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Foreword 


I am pleased to present the September and November volume of my 1999 Report. This Foreword is 
followed by “Matters of Special Importance — 1999” and the Main Points from all of this year’s chapters. Tabling 
of chapters 11 through 19 had been planned for September, but was deferred when the session of Parliament 
ended. This volume of my Report contains the following 23 chapters: 


[ie Agriculture Portfolio — User Charges 

22 Agriculture and Agri-Food Canada — A New Crop: Intellectual Property in Research 

ibe National Defence — Hazardous Materials: Managing Risks to Employees and the Environment 
14. National Health Surveillance: Diseases and Injuries 

iS. Management of a Food-Borne Disease Outbreak 

16. Revenue Canada — Goods and Services Tax: Returns Processing and Audit 

Ikgfe Canada Infrastructure Works Program: Phase I and Follow-up of Phase I Audit 

18. Public Works and Government Services Canada — Alternative Forms of Delivery: 


Contracting for Property Management Services 


19. Industry Portfolio — Investing in Innovation 

20. Fisheries and Oceans — Pacific Salmon: Sustainability of the Fisheries 

aA: Financial Information Strategy: Departmental Readiness 

wie. Attributes of Well-Managed Research Organizations 

23: Involving Others in Governing: Accountability at Risk 

24. The Canadian Adaptation and Rural Development Fund: An Example of Involving Others in Governing 
Zo). Preparedness for Year 2000: Final Preparation 

26. National Defence — The Proper Conduct of Public Business 

OG fe National Defence — Alternative Service Delivery 

28. Canadian International Development Agency — Financial Controls Over Projects 

29. Federal Support of Health Care Delivery 

30. Sole-Source Contracting for Professional Services: Using Advance Contract Award Notices 

31. Department of Foreign Affairs and International Trade — Delivery of Capital Projects in Four Missions 
Cpe Follow-up of Recommendations in Previous Reports 

33. Other Audit Observations 


My Office issued the first volume of the 1999 Report in April, containing 10 chapters. 
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In addition, this year my Office has provided: 


e an auditor’s report and observations on the Financial Statements of the Government of Canada; 


e an auditor’s report and observations on the Statement of Transactions of the Debt Servicing and 
Reduction Account; and 


e some 90 auditor’s reports and observations on federal Crown corporations, federal departmental 
corporations and other entities, territorial governments, corporations and other entities and 
international organizations. 


In 1998-99 my Office completed special examinations of Atomic Energy of Canada Limited, Defence 
Construction (1951) Limited, National Arts Centre Corporation, Via Rail Canada Inc., Pacific Pilotage Authority, 
Canadian Commercial Corporation, Canada Mortgage and Housing Corporation and Business Development Bank 
of Canada. Further, in the subsequent period of April to November 1999, my Office also completed special 
examinations of National Museum of Science and Technology Corporation, Standards Council of Canada, Canada 
Development Investment Corporation, Export Development Corporation, Marine Atlantic Inc., Enterprise Cape 
Breton Corporation and Old Port of Montreal Corporation Inc. 


Under section 11 of the Auditor General Act I may undertake, from time to time, assignments at the 
request of the Governor in Council. No such requests were made in 1999, 


Report of the Auditor General of Canada — November 1999 


Matters of Special 


Importance — 1999 


seve = oe ay 


“irae Sp 
peat s3agtion in! 


cA 


Table of Contents 


Main Points 


Two Cheers for the Nineties 


Introduction 
A decade of great change 
Modernizing government 

The budgetary process: Better but still not good enough 
A long-term perspective is long overdue 


In a surplus environment, unconditional short-term targets based on pessimistic projections 


can lead to adverse consequences 
A multi-year fiscal framework can promote fiscal discipline and sound 
spending decisions 


Without better information on the results of government programs, Canadians 
cannot know what they get for the taxes they pay 

Better cost information is needed for improved performance 

New governance arrangements require new accountability mechanisms 


Revitalization of the public service is critical to the success of all other public 
sector reforms 


Conclusion 


Exhibits 


1 Ratio of Youth and Elderly to Working Age Population 
2 Budget Balance: Forecast vs. Actual 
3. Milestones in Performance Reporting at the Federal Level 


Chapter Supplement: Other Recent and Ongoing Audit Issues 


Managing for sustainable development: Progress has been made but gaps remain 
Managing Atlantic shellfish in a sustainable manner: Have we learned from past mistakes? 
User fees: More than just government revenues 

Grants and contributions programs: Management shortcomings persist 

National Health Surveillance: Lack of collaboration impairs effectiveness 

Millennium bug: no room for complacency 

Opportunities for cost savings 


26 


Report of the Auditor General of Canada — November 1999 


es) 


I eee I a ISS ST 
Matters of Special Importance — 
1999 


Main Points 


As the 1990s are drawing to a close, it is an opportune time to take stock of where we are: to 
reflect on the things we have accomplished over the past decade and the challenges awaiting us in the 
next. In this chapter, I focus on developments within the federal public sector, looking in particular at the 
budgetary planning process, performance reporting practices, financial management systems, new 
governance arrangements and public service renewal. The 1990s witnessed progress in all of these areas, 


but shortcomings remain: 


e The budget process is now more open than it used to be, but the two- to five-year budget 
horizon is much too short to take into account the longer-term implications of the fiscal 
choices we make, especially in the context of an aging population and the fiscal pressures it 
entails. Also, with the expiration of preset funding limits in 1998, there 1s no longer a 
mechanism in place to ensure ongoing review of programs. 


e Nearly five years ago, the government announced a commitment to results-based 
management and reporting. While real progress has been made since then in clarifying 
program objectives and publishing performance measures, departmental reporting 1s still 
oriented mainly toward activities and products, and management is not sufficiently focussed 
on outcomes. 


e The accounting methods and systems now used in government do not provide reliable and 
timely information about what operations or outputs cost. The Financial Information Strategy 
(FIS), a government initiative currently under way, is designed to fill this gap. 
Implementation of this initiative has experienced delays, however. It needs to be expedited, if 
the government’s own target date for this initiative is to be met. 


The delivery of government services is increasingly taking place through non-departmental 
mechanisms, often involving partnership arrangements with other governments or the private 
sector. Since no single minister is wholly responsible for results in such cases, these 
arrangements call for new ways of holding government accountable. Without appropriate 
accountability mechanisms, these arrangements can erode the ability of Parliament to 
scrutinize government and hold it to account. 
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e Government efforts to revitalize the public service have intensified recently, but the job is 
huge and far from finished. Gaps created by years of downsizing and restructuring remain. 
People management practices need to be modernized. And an aging work force portends 
immense staffing needs in the years ahead. Reform and renewal of the public service may 
well be the government’s biggest challenge. 


The 1990s have been a decade of transition in government. As we enter a new decade, we must 
aim to consolidate our achievements and continue moving forward. Building on the foundations of the 
past decade, we can succeed in passing on to posterity a stronger Canada, as our forebears did for us. 
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Vision 
We are committed to making a difference for the Canadian people by 
promoting, in all our work for Parliament, answerable, honest and 


productive government that reflects a commitment 


to sustainable development. 


Mission 


The Office of the Auditor General of Canada conducts independent audits 
and examinations that provide objective information, advice and assurance 
to Parliament. We promote accountability and best practices in 


government operations. 


Elaboration of Mission 


In achieving our mission, 


we want to make a difference by promoting: 


@ a fair and frank accounting of government's stewardship of financial 


and other resources 


® efficiency and productivity in the public service 


¢ 


cost effectiveness of government activities 


@ collection of revenues owed to the Crown. 


Other effects we want to produce 


through our work are: 


® objective assurance on matters found to be satisfactory and 


unsatisfactory 


5 


compliance with authority 


® deterrence of fraud and dishonesty. 
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Two Cheers for the Nineties 


Introduction 


riting in this space in 1995, at the 

midpoint of the 1990s, I wondered 

how the decade would eventually be 
remembered. I speculated that either 
“uncertainty” or “transformation” might describe 
Canada in the 1990s, depending on how we 
responded in the remaining years of the decade to 


the issues generating anxiety at that time: 
constitutional disputes, fiscal difficulties, technological disruptions, 
globalization. 


Looking back now from the perspective of 1999, I think it is fair to say that 

we have made solid progress in tackling these issues. Not that there aren’t 

still concerns. Our journey is not finished. But I believe that, on balance, 
“transformation” will likely trump “uncertainty” in the 


On balance, “transformation” will history books as a byword for the nineties, and the decade 
likely trump “uncertainty” in the will be deemed to deserve two cheers. 

history books as a byword for the In what follows, I concentrate on developments in the 
nineties. federal public sector, an area where, in my view, the 


decade was pivotal. 


A decade of great change 


Canada’s federal public sector has experienced profound changes in 

the 1990s. We entered the decade with huge budget deficits and widespread 
concern that Canada was courting financial crisis. Today, as the decade draws 
to a close, the budget is in surplus and the public debate is over how to use 
the fiscal dividend. The federal government work force, at 220,000 
(excluding personnel in Crown corporations and the military), is 
approximately one-fifth smaller today than it was at the beginning of the 
decade. In December 1990, the government released a blueprint for the 
renewal of the public service, calling for fewer centrally prescribed rules, 
greater freedom for departmental managers to manage their departments and 
more authority for front-line employees. Today these plans are, in large 


measure, fact 


although the journey begun 10 years ago is far from over. 


Similar changes have occurred over the past decade in many other 
jurisdictions, both in Canada and abroad, reflecting the influence of common 


forces. Among these forces are stronger demands on governments to operate 
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efficiently, and new possibilities — created by the information revolution — 


to reorganize the way government works. 


One source of pressure for efficient government is a better informed and 
more demanding public. In part, this is a result of faster communication in a 
wired world. In part also, slow economic growth and rising taxes since the 
mid—1970s have strained household budgets. Families forced to tighten their 
belts to make ends meet are likely to look closely at how 
their governments spend and to insist on better value for 


Improving performance has become 
a pressing need for governments in 


the closing days of the 20th century. The huge buildup of public debt over the past two decades 
adds to the pressure for more efficient government. With a 


money in the provision of public services. 


large proportion of government revenues being diverted 
to servicing the debt, money spent on providing services must go that much 
further to maintain their level and quality without resorting to higher taxes. 
Improving performance has thus become a pressing need for governments 1n 
the closing days of the 20th century. 


At the same time, developments in information technology are creating new 
possibilities for governments to respond to that need. Advances in computing 
and communications are making it easier to disseminate information and 
consult with citizens, fostering more open and consultative government. 
They lower the costs of transacting with outside parties, co-ordinating 
activities at a distance and monitoring performance, thereby favouring new 
forms of service delivery over traditional means. They call for a highly 
skilled and adaptable work force, which requires new and innovative people 
management practices for optimal performance. 


Modernizing government 


These developments are occurring in most Western nations and are 
contributing to a fundamental shift in the way we view government, what it 
does and how it does it. The traditional, bureaucratic model of public 
management — centralized, secretive, hierarchical — is increasingly seen as 
inadequate to the times. While a clear alternative to the traditional model has 


yet to emerge, prominent features of the new generation of public 
management include: 


e amore transparent and forward-looking budgetary process; 


e a decentralized and results-based approach to management: 
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e greater reliance on outsourcing and partnerships with other organizations 
to deliver services; and 


¢ amore incentive-driven and flexible government work force. 


Where does Canada’s public administration figure in this global revolution in 
public sector governance? What changes are needed to ready it for the next 
century? 


These are the questions I take up in this chapter, looking particularly at 
modernized budgetary processes, improved results-based information, new 
accountability mechanisms and public service revitalization. Other issues that 
are not directly related to the theme of modernizing government but that I 
consider important are highlighted in a supplement to the chapter. 


The budgetary process: Better but still not good enough 


The way the government puts its budget together has changed significantly in 
the 1990s. Central to this change have been the pre-budget hearings held in 
the fall by the House of Commons Standing Committee on Finance. I called 
for such hearings in my 1993 Report, and I am pleased to see them become 
part of the regular budget process. In my view, they represent one of the most 
significant budget reforms in decades. 


The hearings provide an excellent opportunity to focus attention on budget 
issues and to involve the public in decisions about budget matters. They 
receive wide publicity, and thus help to improve the public’s understanding 
of the financial constraints and trade-offs the government faces in making 
fiscal policy. They allow the government to test proposals it is considering 
and to receive feedback from the public and from Parliament before 
including such proposals formally in the budget. And they can help mobilize 
public support for change, by including Canadians at large in the decisions. 
The hearings, together with the background documents provided by the 
Department of Finance to support them, have resulted in a much more open 
and transparent consultative process, consistent with best practices in other 
jurisdictions. 


In other respects, however, the federal budget process lags modern practices 
elsewhere. In particular, it is still not sufficiently forward-looking, the focus 
on short-term surplus targets is too narrow and, with Program Review 
completed and immediate fiscal pressures behind us, there is no longer a 


mechanism in place to encourage ongoing review of programs. 


Report of the Auditor General of Canada — November 1999 9 


i 
Matters of Special Importance — 1999 


A long-term perspective is long overdue 


I have long been calling for the government to provide long-term fiscal 
projections as part of the budgetary process. I remain convinced that 
long-term fiscal projections are necessary to counterbalance a natural 
political tendency to focus on the short run. 


In recent years, the government has been producing only 


Long-term fiscal projections are two-year projections. In his fall fiscal update released 


necessary to counterbalance a 


earlier this month, the Minister of Finance for the first 
time provided five-year forecasts of government revenues 


natural political tendency to focus on and expenditures, based on an average of private sector 


the short run. 


forecasts. Extending the fiscal outlook from two years to 

five is an important step in the right direction. But I 
believe that it still falls short of what is needed for a fully informed debate on 
the budget choices we face and the implications that follow from them. For 
that, we need information on Canada’s longer-term fiscal outlook as well. 


Increasingly, other countries are providing the long-term budget frameworks 
that enable legislatures and the public to make informed choices about their 
future. Since long-term projections are necessarily speculative, they should 
not imply a government commitment to the precise outcomes projected. In 
the United Kingdom, for instance, where long-term fiscal projections are 
mandated under the Code of Fiscal Stability enacted last year, these 
projections are clearly labelled as “illustrative”. In New Zealand, where 
similarly long-term projections are required under the Fiscal Responsibility 
Act, the Minister specifies that these projections “are not fiscal forecasts. 
They indicate what the fiscal position might be under alternative 
assumptions.” The value of such projections lies in making the long-term 
consequences of fiscal policy more explicit, both to Parliament and the 


public, and in providing early warning of adverse fiscal trends and lurking 
expenditures. 


In a report tabled in October 1998, the House of Commons Standing 
Committee on Public Accounts endorsed the need for long-term fiscal 
information. In its response to the Committee, the Department of Finance 
argued that long-term projections would undermine “the importance and 
urgency of addressing immediate problems.” However, the Department did 
leave the door open for such projections at some later date, once the 
government has accumulated more annual surpluses and there is less political 
focus on short-term targets. I think the Department is mistaken. In my view, 
the argument that long-term fiscal projections would detract from the 


10 
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importance of immediate problems is not persuasive. Only by considering the 
longer term can we fully appreciate how urgent our present fiscal situation is. 


This is especially true today, given projected demographic trends and their 
consequences for public finances. The ratio of youth and elderly to the 


Exhibit 1 working-age population in Canada is at an 
Ratio of Youth and Elderly to Working Age Population historically low level (see Exhibit 1). This 
favorable ratio will begin reversing in about 10 
Percentage , : 
" years, as the large baby boom generation begins 
reaching normal retirement age. Within three 
75- 


decades, the proportion of elderly in the 
. tN Patent population will have doubled, putting 


unprecedented pressures on our health care and 


ae 


. 


social security systems. Decisions we make today 


J Oilawenre F 5 A ; 
45 ee eee eas old will define the choices available to Canadians not 
re ~~ —_ _ __ - ——__sjust this year and next, but decades ahead. Should 
7 ros we leave that legacy to chance? 
= 65 years and older, - - 
abies Siler wath sad Canada eliminated its deficit much faster than 
15 T T 1 4 4 
‘ . . almost anyone thought likely a few years ago. The 
1980 1986 1992 1998 2004 2010 2016 2022 2028 


Source: Statistics Canada, Matrix 6367 


near-term outlook remains favourable. This 
provides us with a window of opportunity to put 
in place the structures that can withstand the demographic pressures looming 
ahead. To do that, we need information on how serious those pressures are. And 
clearly, such information requires an outlook well beyond the government’s 
current two- to five-year budget horizon. 


In a surplus environment, unconditional short-term targets based on 
pessimistic projections can lead to adverse consequences 


The fiscal plan or framework is a key feature of any modern budget. It reflects 
the government’s fiscal response to current economic conditions and its strategic 
approach to ensuring stability and health in public finances. Until recently, the 
government’s fiscal strategy had focussed exclusively on deficit reduction. With 
the elimination of the deficit two years ago, the government’s announced 
objective now is to maintain a zero balance or a small surplus in its annual 
budget. 


Under the current planning process, specific targets for the budget balance are 
set for each of the two fiscal years following the budget. The government 
commits itself unconditionally to meeting these targets: in the vivid language of 
the Minister of Finance, they are targets to be met “come hell or high water.” To 
ensure that they are, the targets are based on “prudent” economic assumptions, 
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meaning assumptions that interest rates will be higher and economic growth 
correspondingly lower than the consensus forecast by the private sector. 
They also include a $3 billion “contingency reserve” as an additional cushion 


against unforeseen developments. 


As Exhibit 2 shows, over the past five years, the government has in fact 
bettered every target it has set. In the early years of this “prudent” approach 
to budget planning, when the budget was still 
Exhibit 2 in deficit, bettering the target meant, in effect, 
Budget Balance: Forecast vs. Actual moving toward a balanced budget faster than 


($ billion) forecast. This proved useful in earning the 
ion . - . . . 
government a reputation for fiscal discipline 


10 and assuring markets that Canada’s finances 
were at last under control. But now that the 
an budget is in surplus, this approach to budget 


[9] Forecast 
BB Actual 


planning has less benign consequences. 


—10- 


For one, it biases government decisions 
toward higher spending. With budget 


projections based on pessimistic assumptions, 
it is likely that the actual budget surplus will 


exceed the projection. By the time this 


So becomes evident near the year-end, it 1s 
1995 1996 1997 1998 1999 normally too late to affect the result through 
Fiscal Year tax reductions, leaving increased spending as 


the most effective means of eliminating the 
“excessive” surplus. Each of the past two budgets contained significant new 
spending booked in the year the budget was tabled. As a result, recorded 
spending in each of those two years was substantially higher than had been 
planned. In February 1997, program spending for fiscal year 1997-98 was 
budgeted at $105.8 billion, but actual spending recorded for that year was 
$108.8 billion. In February 1998, program spending for fiscal year 1998-99 
was budgeted at $104.5 billion. Actual spending was $111.4 billion, nearly 7 
percent higher. 


At issue here is more than the mere fact that the government spent more than 
it had planned to spend. How this was done also matters. A scramble around 
budget time to find ways to spend money before the year ends is not a 
process calculated to ensure efficiency, effectiveness and economy in the use 
of public funds. In fact, it differs little from end-of-year spending by 


departments with surplus funds, except that it involves not millions but 
billions of taxpayer dollars. 
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The current approach to budget planning also leads to pro-cyclical bias, 
adding stimulus when the economy is strong and removing stimulus when 
the economy is weak. Budget surpluses will tend to be larger than projected 
when economic growth is higher than expected, and smaller when growth is 
relatively low. A policy of sticking to a specific surplus target, regardless of 
events, will therefore normally require increased spending when the 
economy does better than expected and reduced spending when the economy 
does worse. In short, it will tend to stimulate an already strong economy and 
dampen a weak one, exacerbating normal swings in the business cycle. 


The current approach to fiscal planning was adopted at a time of huge 
deficits and widespread skepticism about the government’s ability to rein 
them in — the legacy of a series of overoptimistic forecasts. In those 
circumstances, setting unconditional short-term targets proved a winning 
strategy. With the conditions giving rise to that strategy now behind us, the 
strategy itself needs to be rethought. 


A multi-year fiscal framework can promote fiscal discipline and 
sound spending decisions 


The government’s success in eliminating the deficit was aided by two 

initiatives it launched early in its mandate. The first was Program Review, 

involving a comprehensive examination of existing programs to determine 
whether they were still necessary and, if so, whether they 


The need for ongoing review of could be better run. The second initiative was a new 


ae, ; Expenditure Management System, which capped 
government activities remains. government spending by requiring that all new projects be 


financed with funds reallocated from existing programs. 
Program Review was completed by 1995 and the cap on spending was lifted 
implicitly in 1998, with the elimination of the deficit. The need for ongoing 
review of government activities remains, however, for we still need to ensure 
that they continue to serve a useful public purpose and that they remain 
efficient, effective and affordable. 


A multi-year fiscal framework can provide the incentive for this ongoing 
review. As part of such a framework, the government would establish 
aggregate spending limits consistent with the fiscal future it wants to bring 
about. Limits on total spending can help promote good management and 
fiscal discipline in the public sector. They play the role that household 
budgets do in family spending decisions: forcing decision makers to 
recognize the costs of their decisions and the need to choose carefully among 


their spending options. 
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There is no limit to the demands that can be made of government or to the 
services government can choose to provide. In the absence of a budget 
constraint, the temptation to do too much is natural. When spending on new 
programs can be accommodated without depriving existing ones of 
resources, why bother with the difficult — and costly — exercise of carefully 
analyzing new proposals, or risk alienating political support by opposing 
them? Government ministers and managers have limited 
incentive to curb their own spending, or to oppose that 


Even in good times — especially in of others, when it does not interfere with their own 
good times — it is important that we favourite programs. Establishing explicit limits on total 


set affordable limits on spending and 


live within them. 


spending changes this dynamic, by bringing home to 
decision makers the fact that every program has to 
compete for resources with all others. It encourages a 
more careful review of new initiatives and comparison 
of their merits with those of other initiatives that would have to be sacrificed 
to put them into place. 


Canada’s finances improved significantly over the past few years. Our 
challenge now is to maintain the fiscal discipline that made those 
improvements possible. Even in good times — especially in good times — it 
is important that we set affordable limits on spending and live within them. 
As we learned in the early 1990s, if we do not establish limits ourselves, the 
financial markets will do it for us. The whole point of a good budgetary 
process is to prevent us from ever falling back into that condition. 


For related OAG work, see: 1998 Chapter 6 Population Aging and 
Information for Parliament: Understanding the Choices; 1995 Chapter 9 
Information for Parliament — Deficits and Debt: Understanding the 
Choices; 1993 Chapter 5 Information for Parliament — Understanding 
Deficits and Debt. 


Without better information on the results of government 


programs, Canadians cannot know what they get for the 
taxes they pay 


In my first report as Auditor General eight years ago, I stressed the need for 
managers and departments to account for results: “for what they have — or 
have not — achieved with taxpayers’ money.” Information on results is 
necessary first and foremost because it goes to the heart of why organizations 
exist, which of course is to achieve desirable results, not to just use up 
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resources. But it is necessary for practical reasons as well — more 


specifically, to: 


e manage programs and services more efficiently and effectively. We 


cannot improve what we do unless we know the results of what we are 


doing; 


e support better allocation of public resources. Intelligent decisions on the 


funding of programs require information on what those programs are 


achieving or are not achieving; and 


e improve accountability in the use of public funds. As delegation of 


authority to use public resources proceeds, clearer reporting on 


objectives and on results achieved becomes all the more necessary to 


preserving democratic control and accountability for use of the 


taxpayers’ money. 


The federal government has introduced several important initiatives in this 


area over the past few years, as Exhibit 3 shows. These initiatives have 


Exhibit 3 


Milestones in Performance Reporting at the Federal Level 


February 1995. The government announced a new Expenditure 
Management System, stressing commitment to “performance 
information and management.” 


November 1995. In his annual report, entitled Strengthening 
Government Review, the Treasury Board President reaffirmed the 
government’s commitment to results-oriented management and set 
out its strategy to bring it about. 


April 1997. The House of Commons approved a motion to split 
the annual Estimates Part II] into two documents, a Report on 
Plans and Priorities, tabled in the spring, and a Performance 
Report, tabled in the fall. The intention is for the reports on plans 
and priorities to establish performance expectations and for the 
performance reports to disclose how well previous expectations 
were achieved. The first complete set of Performance Reports was 
tabled in November 1997 and the first full set of Reports on Plans 
and Priorities in March 1998. 


certainly made a difference. Information to 
Parliament is now more complete and 
accessible; descriptions of departmental 
missions, structures and activities are better; 
and progress has been made in setting 
expectations and reporting performance. 


Yet, as the government acknowledges, much 
remains to be done. Objectives in departmental 
plans are frequently vague. The specific results 
expected are often neither clearly stated nor 
measurable, and they tend to focus on outputs 
(what is produced), not outcomes (what 
happens as a result). Reporting is still oriented 
mainly toward activities and products. 


It was recognized from the beginning that 
managing for and reporting on results would 
be difficult to implement. Establishing 


meaningful performance expectations, reliably measuring accomplishments 


and reporting on them fairly are not easy tasks. Causal links between 


program outputs and societal outcomes are rarely unambiguous or precise. 
Our expectations about what is possible must therefore be realistic. 


That said, however, I’m convinced that focussing on results is the right 
approach. Other jurisdictions that have travelled this road ahead of us — 


Report of the Auditor General of Canada — November 1999 15 


je aa 
Matters of Special Importance — 1999 


Australia, New Zealand, the United Kingdom, the United States, the Nordic 
countries — acknowledge the difficulties, but none that I know of is thinking 
of turning back. Independent standard-setting bodies in Canada agree. 
CCAF-FCVI Inc., the Canadian Institute of Chartered Accountants, the 
Canadian Council of Legislative Auditors — each one of these organizations 
has recently sponsored major research and development programs aimed at 
promoting performance reporting in the public sector. These initiatives 
reflect a growing consensus among practitioners on the importance of 
results-based reporting for effective performance and accountability in the 


public sector. 


Nevertheless, progress in this area will likely remain slow until appropriate 

incentives to report on results and performance are put in place. An 

important first step would be for the government to find a way to make better 
use of performance information in its decisions on 


Progress in this area will likely remain _ planning and allocating resources among programs. 


slow until appropriate incentives to 


Parliamentarians can also help by reviewing 
Departmental Performance Reports more closely and 


report on results and performance are using them more extensively in scrutinizing government 


put in place. 


activities. And departmental managers must be 

encouraged to integrate performance measures more fully 
in their own planning decisions. For all of this to happen, strong support 
from senior managers is needed to change the culture to one focussing on 
outcomes rather than only inputs and outputs. Without this support, progress 
will be very slow. With it, public management can be modernized. 


For related OAG work, see: 1998 Report on Parliamentary Committee 
Review of the Revised Estimates Documents; 1997 Chapter 5 Reporting 
Performance in the Expenditure Management System; 1997 Chapter 11 
Moving toward Managing for Results; 1997 Chapter 22 Crown 
Corporations: Making Performance Measurement Work; 1995 Chapter 10 
Crown Corporations: Fulfilling Responsibilities for Governance; 1993 
Chapter 4 Crown Corporations: Accountability for Performance. 


Better cost information is needed for improved 
performance 


Good performance reporting and managing for results are also hampered at 
present by poor information on the cost of government activities. Accurate 
and timely financial information is essential to an effective 
performance-based system. To assess performance, results must be linked to 
costs. It is not enough to know merely what is produced; we also need to 
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know the cost of that production. The accounting methods and systems now 
used in government do not provide reliable and timely information about 
what operations or outputs cost. 


The government’s Financial Information Strategy (FIS) 


The accounting methods and systems currently being implemented is designed to fill this gap. 


now used in government do not Among other things, FIS will replace the somewhat 


provide reliable and timely archaic, partly cash-based present system of accounts 
‘ : ’ with full accrual accounting similar to that used by 
information about what operations OF business firms. This means that capital asset costs will 
outputs cost. be recognized over time as the assets are used up, 
etn sein > instead of all “up front” in the year acquired. By 
appropriately associating costs with the operations-they support, 
accrual-based accounting will provide us with better information on what 


government programs actually cost. 


First launched in 1989, FIS was re-endorsed as a government priority 

in 1995. The target date for implementation across government is April 2001. 
FIS will bring the management of government into alignment with modern 
private sector practices in other jurisdictions, such as Alberta, Australia, New 
Zealand and the United Kingdom. 


The pace of implementation will have to accelerate if that target date is to be 
met. As I report in Chapter 21, FIS remains far from completion. At the time 
of our audit, most government departments were ill-equipped to implement 
FIS, and central management and support by the Treasury Board Secretariat 
needed considerable strengthening. In my view, the momentum created 
across government to deal with Y2K must now be carried forward to FIS. In 
particular, new financial systems and accounting rules must be put in place 
within two years if the present deadline is to be met, and officials throughout 
government must be encouraged to use the more businesslike and timely 
financial information that FIS will provide for day-to-day management and 
accountability. 


The government cannot afford to let this project lag, especially in light of 
previous delays. Not only meaningful performance reporting but program 
evaluation, inventory and asset management, decisions about how to fund 
and run programs, how much to charge for user services where user-pay 
policies are in place — all of these activities require information that the 
government does not now have and that FIS, properly implemented, can 
provide. 


For related OAG work, see: 1999 Chapter 21 Financial Information 
Strategy: Departmental Readiness; 1998 Chapter 18 The Financial 
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Without appropriate accountability 


Information Strategy: A Key Ingredient in Getting Government Right; 1997 
Chapter 2 Financial Management: Developing a Capability Model; 1997 
Chapter 3 Management of the Government's Accounting Function: A Central 


Agency Perspective. 


New governance arrangements require new accountability 
mechanisms . 


In Chapter 23 of this Report, I discuss the results of a government-wide audit 
of accountability in new types of governance arrangements that involve 
organizations outside the federal government in the delivery of federal 
objectives. The Labour Market Development Agreements with the provinces, 
the Canada Foundation for Innovation, the National Child Benefit, the 
Canada Millennium Scholarship Foundation and the St. 
Lawrence Seaway Management Corporation are 
examples. There has been a significant growth in the use 


and good governance mechanisms, of such arrangements and they hold promise of more 
these arrangements can erode the effective and efficient delivery of federal programs and 


ability of Parliament to scrutinize the 


use of federal power. 


services. 


I am concerned, however, that the government has not 

given proper attention to the implications for 
accountability and good governance. By their very nature, these 
arrangements challenge the traditional accountability relationship that sees 
ministers answerable to Parliament for their policies and programs and, 
through Parliament, to citizens at large. Since other parties are also involved 
in these arrangements, ministers are never wholly responsible for them. In 
some Cases, arrangements have intentionally been set up to be totally 
independent from ministers, even though they may depend on federal funds 
and federal authority. Without appropriate accountability and good 
governance mechanisms, these arrangements can erode the ability of 
Parliament to scrutinize the use of federal power and the right of citizens to 
accountable government. 


Our government-wide audit found significant weaknesses in existing 
accountability mechanisms. Arrangements have been established without 
adequate provision for Parliament to scrutinize the way they operate or to 
hold the government to account for their performance. And they do not 
always provide means for public input, citizen redress, or recourse when the 
public’s expectations are not met. Indeed, there are few mechanisms in place 
that would allow the government itself to monitor these arrangements 
effectively. As things now stand, the government does not know to what 
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extent it is using these arrangements or whether, as instruments of federal 
policy, they are performing well or failing. 


Fiscal and technological forces are pushing governments to use innovative, 
non-traditional ways of delivering programs and services. Reliance on new 
governance arrangements is therefore likely to grow in the future. As we 
move to these new forms of delivery, however, we should be careful not to 
jettison fundamental principles of parliamentary democracy along the way. 
Whenever the government enters into new arrangements for program 
delivery, it must ensure that they address several key elements: credible 
reporting, effective accountability mechanisms, adequate transparency and 
protection of the public interest. Parliament itself needs to consider carefully 
what governing framework it expects in such new arrangements. While 
increased reliance on new forms of public service delivery may be inevitable, 
it need not be at the expense of accountability to Parliament and the public. 


For related OAG work, see: 1999 Chapter 5 Collaborative Arrangements: 
Issues for the Federal Government; 1999 Chapter 6 Human Resources 
Development Canada — Accountability for Shared Social Programs: National 
Child Benefit and Employability Assistance for People with 

Disabilities; 1999 Chapter 23 Involving Others in Governing: Accountability 
at Risk; 1999 Chapter 24 The Canadian Adaptation and Rural Development 
Fund: An Example of Involving Others in Governing; 1998 Discussion Paper 
Assessing Alternative Service Delivery Arrangements (June); 1998 Chapter 
12 Creation of the Canadian Food Inspection Agency; 1997 Chapter 5 
Reporting Performance in the Expenditure Management System; 1997 
Chapter 19 Transport Canada — The Commercialization of the Air 
Navigation System. 


Revitalization of the public service is critical to the 
success of all other public sector reforms 


A highly competent and well-motivated government work force is key to 
successful public sector reform. In the end, it is people who must implement 
the reforms, deliver the services and provide feedback for 
or : efeuels hat Be improved performance. Modernizing the public service is 
Modernizing the public service is vital A H y . Lacey 
therefore vital to modernizing government. Bringing this 


to modernizing government. about may be the government's biggest challenge. 


The public service today is a knowledge-based 

institution: a majority of its members are in the 
professional or management categories. In the future this will likely be even 
more pronounced. Human resource systems and practices designed for an 
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earlier era will not sustain the public service of the future. “Knowledge 
workers” tend to be self-directed. They perform better in an environment of 
incentives than of rules. They demand greater involvement in decision 
making. They value flexible working arrangements and opportunities for 
continuous learning. Smart companies are already operating this way. 
Canada’s public service will have to learn to compete with them, if it is to 
attract and retain the right workers for the future. 


At the same time, the public service is an institution under considerable 
pressure. Work force reductions and years of restraint and restructuring have 
had an unsettling impact. Skills and competencies to run programs are 
lacking in many areas. According to recent surveys, increased workloads and 
poor career paths are major concerns among federal public service 
employees, while young Canadians no longer view the public service as an 
attractive career option. The government will have to 


Failure to reform the public service cope with these immediate concerns while establishing a 


would put all other public sector 


reforms at risk. 


plan to deal with the longer-term needs of the public 
service. 


One issue that is both immediate and longer-term is work 
force renewal. The public service work force is older than the Canadian work 
force as a whole. A large proportion of public service employees will likely 
retire during the next decade. Over one half of senior managers will be 
eligible for retirement within the next five years. The public service will have 
to replace these employees at a time of generally slow labour force growth 
and increasing demands for skilled workers in other sectors of the economy. 
And it will have to do so while safeguarding core public sector values like 
integrity, non-partisanship, diversity and respect for due process. 


The government is aware of these problems and the challenges they pose. It 
has paid more attention lately to human resource issues and progress has 
been made in some areas, as I reported last April in a chapter on the 
management of science and technology personnel. New recruiting and 
development programs, resumption of collective bargaining, la Reléve and a 
universal job classification system are among the prominent recent 
government efforts aimed at adapting human resource management practices 
to the current and anticipated needs of the public service work force. A host 
of other, less visible projects with a similar purpose are under way across the 
public service and within individual departments and agencies. But the job is 
huge, and progress appears frustratingly slow. 


It is likely to remain so until the government succeeds in establishing a 
vision of the public service that can give all these efforts coherence and 
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direction. What is the proper balance between central direction and 
departmental autonomy over personnel management? How do we maintain a 
unified public service while devolving responsibility to departments and 
agencies? Which government functions should remain in the core public 
service and which could be allowed to migrate elsewhere? Without clear 
answers to such questions, progress in reforming the public service will 
likely stall. And failure to reform the public service would put all other 
public sector reforms at risk. 


For related OAG work, see: 1999 Chapter 9 Management of Science and 
Technology Personnel: Follow-up; 1998 Chapter 1 Expenditure and Work 
Force Reductions in the Public Service; 1998 Chapter 2 Expenditure and 
Work Force Reductions in Selected Departments; 1997 Chapter 1 
Maintaining a Competent and Efficient Public Service. 


Conclusion 


The 1990s have been a decade of transition in government, both in Canada 
and elsewhere. The revolution in information technologies that is forcing 
companies across the spectrum to “reinvent” themselves is having a similar 
impact on governments. Like corporations in the private sector, governments 
are becoming smaller, mission-oriented organizations providing goods and 
services through varying arrangements with other parties. The principle that 
best describes this emerging public sector model is captured by the maxim 
that “governments should steer, not row.” 


Common elements of this emerging model include greater reliance on means 
outside of departments to deliver services, a shift in focus from inputs and 
outputs to outcomes, management based on results rather 


The federal government has made than rules, and a lessening of centralized controls in 

: : : favour of increased managerial autonomy and employee 
eile) Strides during the decade empowerment. It also encompasses a more participatory 
now ending. But much more still form of governance, entailing more transparency in 
needs to be done before the results government operations and more consultations with the 
merit three cheers rather than two public in formulating policies and making decisions. 


In most of these areas, the federal government has made 
important strides during the decade now ending. But much more still needs 
to be done before the results merit three cheers rather than two. An open and 
forward-looking budgetary process, good information on the results of 
government activities and integration of that information in management 
decisions, a clear perspective on the role and make-up of the future public 
service along with the actions necessary to see this brought about — it will 
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take improvements in all of these areas. In most cases, all that is needed is 
further progress on processes now in place. It is therefore certainly within 
our reach — and we should make it our goal — to earn that third cheer 


before the next decade is out. 


At the threshold of a new century, Canada stands tall and full of promise. As 
our membership in the G-7 shows, Canada ranks among the economically 
most important countries in the world today. And our first-place ranking in 
the UN Human Development Index for six years in a row indicates that our 
achievements extend beyond the economic sphere alone. 


National success is not foreordained, however. As we know from the 
experience of other countries, it is possible for societies to regress as well as 
to move forward. Which course they find themselves on depends critically on 
the performance of the public sector: its capacity to generate informed 
policies, provide the necessary infrastructure, minimize the burdens it places 
on private firms and individual taxpayers. To ensure that we move forward, 
we need government institutions that are responsive, effective and efficient. 
Building on the foundations of the past decade, we can succeed in passing on 
to posterity a stronger Canada, as our forebears did for us. 
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Chapter Supplement: Other Recent and Ongoing Audit Issues 


Managing for sustainable development: 


Progress has been made but gaps remain 


It has been four years since the Auditor General Act 
was amended to help strengthen the federal 
government's performance in protecting the 
environment and promoting sustainable development. 
Those amendments required government departments 
to prepare sustainable development strategies, and 
created the position of Commissioner of the 
Environment and Sustainable Development within my 
Office to monitor and report annually to the House of 
Commons on departmental performance. 


Since then, the Commissioner of the Environment and 
Sustainable Development has reported three times to 
the House of Commons on a total of 17 audits and 
studies of issues with an important environmental or 
sustainable development component. What have we 
learned? 


Many environmental and sustainable development 
issues are, by their very nature, difficult to manage. 
They are often scientifically complex, involve long 
time frames and do not fit neatly within a single 
department’s or government’s mandate or jurisdiction. 


Notwithstanding those difficulties, when policies and 
programs are developed and announced, the public 
does expect that they will be implemented and that 
intended results will be achieved. But as the 
Commissioner stated in his May1999 Report, the 
federal government’s management of environmental 
and sustainable development issues is a mix of 
successes, failures and works in progress. 


Both the Commissioner and I are convinced that 
performance can be significantly improved by 
applying sound management practices to the 
sustainable development agenda. Departments and 
agencies have taken the first steps: we have seen 
examples of good environmental management, and a 
growing interest among public servants in 
understanding and sharing good practices. 


But we still have a way to go before those good 
practices become the norm and are applied routinely to 
the management of environmental and sustainable 
development issues. 


For related OAG work, see: 1999 Report of the 
Commissioner of the Environment and Sustainable 
Development. 


Managing Atlantic shellfish in a sustainable 


manner: Have we learned from past mistakes? 


Last April, I reported the results of an audit of the way 
the Department of Fisheries and Oceans manages the 
Atlantic shellfish fisheries. The findings of that audit 
lead me to question whether government has learned 
the lessons of past failures in fisheries management. 


Many of the management problems found in previous 
audits (reported in Chapters 14 and 15 of my 1997 
Report) continue to exist. Of greatest concern is that the 
Department has still not developed an overall 
management framework for the Atlantic fisheries. As a 
result, fundamental questions about fisheries 
management remain unanswered. What are the overall 
objectives for managing the fishery? How are 
competing biological, social and economic factors to be 
balanced in decisions made by fisheries managers? 
How can decisions be made in an open and transparent 
manner, consistent with a set of guiding principles for 
the fishery? In the face of inevitably limited information 
about the resource, how is an appropriate element of 
caution introduced in regulatory policies about the 
fishing industry? These are only a few of the key issues 
that the Department must address. 


I believe that the Department must move quickly to 
develop an overall framework for managing Atlantic 
fisheries in a sustainable manner. The Standing 
Committee on Public Accounts also made such a 
recommendation in a report to the House of Commons 
last June. This framework should be the foundation on 
which future decisions are based. Without this 
foundation, the long-term sustainability of the resource 
remains at risk. 


For related OAG work, see: 1999 Chapter 4 Fisheries 
and Oceans — Managing Atlantic Shellfish in a 
Sustainable Manner; 1999 Chapter 20 Fisheries and 
Oceans — Pacific Salmon: Sustainability of the 
Fisheries; 1997 Chapter 14 Fisheries and Oceans 
Canada — Sustainable Fisheries Framework: Atlantic 
Groundfish; 1997 Chapter 15 Fisheries and Oceans 
Canada — Rationalization to Renewal: Atlantic 
Groundfish. 
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User fees: More than just government 


revenues 


The idea of government user fees is not new; some 
federal government departments have been charging 
since the turn of the century for services they provide. 
But only in this decade has the idea received serious 
widespread attention, particularly following the 1995 
Program Review when departments revamped 
programs to deal with the deficit and to improve their 
services to Canadians. Existing user fees were 
increased and new ones introduced. 


Critics of this initiative have been many and vocal. 
User groups, and respected commentators, have 
criticized the way user charges have been implemented 
and questioned the merits of the policy. One of the 
main architects of the Program Review exercise argues 
that managers relied too heavily on user charges to 
meet Program Review objectives. An influential 
industry group has called for a fee moratorium across 
government. 


Our audit work, both in past years and this year, 
provides ample reason for concern about how user 
charges have been implemented. Weaknesses in 
determining costs of individual services have left users 
with an unsettling feeling about the cost figures used 
to set fees. Often, managers can’t make a convincing 
case that users are not being charged for inefficient 
operations. Despite the fact that user charges are a 
contentious issue that has sparked a great deal of 
debate, departments and the Treasury Board 
Secretariat have generally reported little information to 
Parliament and the public on plans for user charges or 
on results. 


What is to be done, then? The Treasury Board 
Secretariat has promised to review experience with the 
policy this year. With many voices arguing that user 
charges should be frozen or abandoned, it is important 
to consider the arguments for fixing the initiative. Our 
work provides evidence that user charges can generate 
additional benefits, beyond revenues, to both the 
government and users of services. This can serve as an 
incentive for users to work with managers and focus 
services on what they truly value. The pressure that 
user charges create to improve service standards, 
costing systems and efficiency of operations should be 
welcomed by users and government officials alike. 
User charges are not applicable everywhere, nor 
should they be seen as a panacea, but they can play a 
valuable role in the management of government 
services. 

For related OAG work see: 1999 Chapter 11 
Agriculture Portfolio: User Charges 


Grants and contributions programs: 


Management shortcomings persist 


Grants and contributions are transfer payments from the 
government to individuals and organizations of various 
types. Grants are unconditional transfers; that is, they 
impose no requirements on recipients beyond the initial 
entitlement conditions. Contributions, however, are 
transfer payments that are subject to performance 
conditions specified in the contribution agreements. 


Over the past two decades, my Office has carried out 
several audits of the management of grant and 
contribution programs by federal departments and 
agencies. These audits identified persistent 
shortcomings, ranging from problems in compliance 
with program authorities to weaknesses in program 
design, and to poor controls and inadequate reporting 
on performance. 


I can’t help but express a certain degree of frustration 
with the management of grant and contribution 
programs. I believe that my Office has made a 
difference in the programs that we have audited. What 
concerns me is that when we look at programs we have 
not audited before, we find similar problems. I know 
that parliamentarians share my concern. For instance, in 
its 24th report to the House of Commons tabled in 
March 1999, the Standing Committee on Public 
Accounts recommended that my Office consider 
undertaking an audit of the management of grant and 
contribution programs government-wide and report our 
conclusions to Parliament. 


In my response to that recommendation last June, I 
informed the Committee that we were planning such an 
audit for reporting in the fall of 2001. Our intent is to 
look at a number of different departments and agencies 
with relatively large grant and contribution programs, 
so that we can make summary observations on the 
quality of the management of such programs across 
government. 


In the meantime, we continue to carry out audits of 
specific grant and contribution programs and report on 
each to the House of Commons. This year, we 
examined payments made under The Atlantic 
Groundfish Strategy, Phase II of the Canada 
Infrastructure Works Program, and four grant and 
contribution programs in the Industry Portfolio. 


For related OAG work, see: 1999 Chapter 7 The 
Atlantic Groundfish Strategy: Contributions and 
Grants; 1999 Chapter 17 Canada Infrastructure Works 
Program: Phase II and Follow-up of Phase I 

Audit; 1999 Chapter 19 Industry Portfolio — Investing 
in Innovation; 1998 Chapter 27 Grants and 
Contributions: Selected Programs in Industry Canada 
and Department of Canadian Heritage. 
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Health surveillance is a core function of public health. 
In Chapter 14 of this Report, we examine the way 

Health Canada carries out national health surveillance 
and control activities, and how those activities support 
the other components of the public health framework. 


The chapter identifies weaknesses in national 
surveillance of diseases and injuries that, taken 
together, have clear national implications for public 
health. First, they compromise Health Canada’s ability 
to detect, prevent and control health risks associated 
with outbreaks of communicable diseases and other 
health threats. Second, they compromise its ability to 
plan and carry out programs that deal with the causes 
and treatment of diseases. 


In a separate but related chapter, we present a case that 
illustrates many of the issues raised in Chapter 14. We 
looked specifically at how federal and provincial 
agencies — Health Canada’s Food Directorate with its 
Laboratory Centre for Disease Control (LCDC), the 
Canadian Food Inspection Agency (CFIA), and 
provincial and local public health departments — 
responded to an outbreak of a food-borne disease that 
occurred in March and April 1998. 


We found that some important elements of the 
response to the outbreak worked well, but others did 
not; for instance, there was insufficient co-operation 
among the agencies involved. The CFIA did not share 
certain information when requested by provincial 
public health officials to assist in the investigation. 


A formal framework is needed that sets out clearly the 
roles and responsibilities of the LCDC in relation to 
those of other participants, in order to guide the 
response to threats to public health. Many individuals 
could have avoided this illness had the federal and 
provincial health departments acted more quickly. 


For related OAG work, see: 1999 Chapter 14 National 
Health Surveillance — Diseases and Injuries; 1999 
Chapter 15 Management of a Food-Borne Disease 
Outbreak; 1999 Chapter 29 Federal Support of Health 
Care Delivery. 
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Millennium bug: no room for complacency 


Year 2000, the two-digit year code problem (also 
known as Y2K and the millennium bug) has been a 
cause of concern to industry and governments around 
the world. The estimated costs of addressing this 
problem run as high as US $800 billion worldwide. In 
August 1999, the federal government estimated the 
costs of its Year 2000 projects at $2.2 billion. 
According to the Treasury Board Secretariat, the final 
costs could reach $2.5 billion. However, given the 
extensive reliance on computerized processes by both 
industry and government, the implications of not 
addressing the problem adequately could be immense. 


We audited the government’s preparedness for the 
millennium bug in each of the past three years. 

Our 1997 Report noted our concern about the slow 
pace of Year 2000 work. In 1998, we remained very 
concerned that some essential services could be 
interrupted in 2000. In 1999, as we completed our 
work in individual departments and agencies, we 
reported our findings to management and suggested 
actions to consider. That additional step was taken to 
provide more time for those organizations to act. 


The government has made significant progress in 
preparing its systems that support government-wide 
mission-critical functions for the Year 2000. In 

July 1999, work on those systems was 99 percent 
complete. The remaining work was scheduled to be 
completed by October 1999. 


Departments have also been preparing for potential 
system failures that may be caused by Year 2000. 
However, though contingency procedures have been 
defined, we found that contingency planning needs 
more work. In the plans we reviewed, some key 
components were incomplete or lacking specific 
details. 


As we completed our audit in September 1999, much 
had been accomplished to prepare for the Year 2000 
but work still remained to be done. We concluded that 
the government needs to remain vigilant to minimize 
potential disruptions in January 2000 and beyond. 


For related OAG work, see: 1999 Chapter 25 
Preparedness for Year 2000: Final Preparation; 1998 
Chapter 20 Preparedness for Year 2000: 
Government-Wide Mission-Critical Systems; 1997 
Chapter 12 Information Technology. Preparedness for 
Year 2000. 
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Opportunities for cost savings 


Improving the efficiency and effectiveness of 
government activities and cultivating a healthy regard 
for economy are objectives central to our mandate and 
to the work that we do at the OAG. The impact of our 
work is not easily quantifiable in most cases, but often 
in the course of our audits we find opportunities for 
cost savings to which a price tag can be attached. 
Following are two recent examples: 


Materiel management. Government holdings of 
materiel (assets other than money and real property) 
exceed $50 billion. Our 1996 audit of how these 
holdings are managed identified significant 
shortcomings, resulting in poor value for money and 
excess inventories. Our report recommended action to 
improve information on materiel holdings and rectify 
deficiencies in materiel management systems. In a 
follow-up to our audit, reported last December, we 
found that departments had made significant progress 
in materiel management. Since 1996, inventory 
holdings had been reduced by more than $480 million. 
This may reduce annual carrying costs by as much as 
$100 million. 


Fiscal equalization payments. Our 1997 audit of the 
Equalization Program questioned the inclusion of user 
fees among the set of taxes forming the base of the 
equalization formula. The basic purpose of 
equalization transfers is to reduce disparities among 
provinces in their fiscal capacities to provide public 
services. The audit pointed out that to the extent that 
government-provided services are financed directly 
through user fees, the users of those services also 
generate the capacity to provide them: there is no need 
for transfers to support provincial capacities in such 
cases. Looking at this question another way, fees used 
to finance “user pays” activities are not available to 
finance general government services. There is no 
revenue source, as such, to equalize. In reforms to the 
equalization formula introduced earlier this year 
following consultations with the provinces, the 
government amended the equalization formula to 
exclude half of the user fees from the tax base. This 
change will be phased in over five years. When fully 
implemented, it will save the government 
approximately $350 million annually. 


For related OAG work, see: 1999 Chapter 32 
Follow-up of Recommendations in Previous Reports: 
Department of Finance — Equalization Program; 
1998 Chapter 28 Follow-up of Recommendations in 
Previous Reports: Materiel Management in the 
Federal Government; 1997 Chapter 8 Department of 
Finance — Equalization Program; 1996 Chapter 23 
Materiel Management in the Federal Government. 
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Correctional Service Canada 


Reintegration of Offenders 


Chapter 1 - Main Points 


D1 Correctional Service Canada has made a concerted effort to respond to our 1994 and 1996 observations 
concerning the management of its offender reintegration activities. However, there are some important areas that 
require further improvement. 


12 The Service is now moving in the right direction. It has recently implemented change initiatives in 
several areas. Among them, it has strengthened the ability of national headquarters to direct and co-ordinate 
offender reintegration activities Service-wide; implemented a major initiative to streamline its reintegration 
operations; achieved international recognition for some of its offender rehabilitation programs; and improved its 
ability to measure the results and performance of its reintegration activities. 


ibs) Progress notwithstanding, improvement is still needed in some key areas: 
e more timely acquisition of official documents for initial offender assessment; 
e more timely casework preparation to meet the offender’s first parole date; 
e aclear operational strategy for offender employment programs; 
e _ better-quality offender reintegration reports for the National Parole Board; and 


e improved adherence to national standards for frequency of contact with offenders in the community. 


Background and other observations 


1.4 Correctional Service has as one of its main responsibilities the safe reintegration of offenders into the 
community. This entails assessing offender risk and needs; preparing the offender for release into the community: 
reassessing offender suitability for release and making a recommendation to the National Parole Board; and 
providing supervision and programs for offenders in the community until the end of the sentence. 


is Overall spending on reintegration has risen by $38 million (13 percent) over the past three fiscal years. 
Correctional Service Canada spends about $329 million or 28 percent of its total expenditures for the reintegration 
of offenders. 


1.6 In 1997-98, there were 13,449 incarcerated offenders in federal institutions and 8,744 offenders in the 
community, most of whom were supervised by Correctional Service parole officers. Until recently, the proportions 
of federal offenders in institutions and under community supervision have remained fairly constant. However, in 
1997-98, the number of offenders supervised in the community increased by about 500. 


1.7 Under the Corrections and Conditional Release Act, there are several different ways that an offender can 
be released into the community: day parole (six months prior to full parole); full parole (at one third of the 
sentence) and statutory release (after two thirds of the sentence). Some offenders will be detained until the end of 
their sentence. 

1.8 A recent change allows offenders serving their first federal sentence who have not been convicted of a 
violent crime or serious drug offence to be released on day parole at one sixth of their sentence (accelerated parole 
review). 
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1.9 Our previous audit work, in 1994 and 1996, identified systemic weaknesses in the Service’s management 
of reintegration activities. Those audits identified concerns in such areas as work standards, quality assurance 
procedures, performance information, implementing basic changes and learning from successes and failures. 

This chapter revisits those issues. 


1.10 As promised to the Public Accounts Committee in April 1998, we reviewed changes made by the 
Service to the custody rating scale and looked at whether the Service had implemented the new offender security 
reclassification instrument. We found that the changes to the custody rating scale reduced overrides with minimal 
impact on the number of escapes. The Service has just implemented a new reclassification instrument, as 
promised. 


1.11 In addition to their institutional security responsibilities, senior correctional officers still do not 
consistently perform their required offender reintegration duties, a necessary input to offender assessment reports 
to the National Parole Board. The Service has undertaken an initiative to address this issue. 


1.12 While the Service has developed a continuum of rehabilitation programs from the institution to the 
community, its ability to deliver these programs to offenders in the community falls short of current needs. 
Research indicates that many intervention programs that deal with offenders’ criminogenic needs are more 
effective when delivered in the community. 


Correctional Service’s responses to our recommendations are included in this chapter. The Service concurs 
with the recommendations made and its responses indicate its commitment to take the necessary corrective 
action. 
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Revenue Canada 


Underground Economy Initiative 


Chapter 2 - Main Points 


2.1 Revenue Canada has reported that the tax impact (taxes resulting from enforcement actions) of its 
activities to address the underground economy was $2.5 billion over five years; however, this includes the results 
of both regular ongoing enforcement programs and the Underground Economy Initiative. The actual tax impact 
attributable to the detection of unreported income by the 1,000 staff allocated to Initiative audit activities is much 
less than the $500 million reported. 


2.2 It is difficult to assess the overall success of the Initiative in combatting the underground economy 
because the Department does not measure and report on the full range of Initiative activities and how they have 
changed taxpayer behaviour. 


23 Revenue Canada needs to assess the role that social marketing might play in making the public aware of 
the societal costs of unpaid taxes and in soliciting its support to combat the underground economy. Polls indicate 
that an alarming number of Canadians would be willing to participate in the underground economy. The 
Department also needs to strengthen the activities that promote voluntary compliance by businesses. 


2.4 The underground economy is a difficult and complex problem to solve and the size of the tax loss is 
significant. The problem requires continuous attention and sustained efforts from Revenue Canada and all 
Canadians. 


Background and other observations 


2.9 The underground economy results in tax evasion and represents an estimated annual loss of federal and 
provincial tax revenues of $12 billion. Tax evasion is not a victimless crime. It puts honest businesses at a 
competitive disadvantage and, in some cases, out of business. It also causes honest taxpayers to bear the tax load 
of those who cheat. If left unchecked, underground economy activity could lead to a loss of faith in the fairness of 
Canada’s tax system. 


2.6 In 1993, Revenue Canada announced a new initiative to combat the underground economy by allocating 
200 staff to its non-filers and non-registrants program and 1,000 staff to the audit of small businesses, where most 
of the underground economy activity exists. Thirty-five percent of the Department’s audit staff for small and 
medium-sized businesses are now involved in the Underground Economy Initiative audit activities. 


2.7 The Initiative as planned was, in our view, a balanced approach to combatting tax evasion in the 
underground economy. It included activities to promote voluntary compliance in small businesses such as 
community visits and consultations with industry associations. As well, it involved other federal departments, 
provincial and municipal governments and private sector organizations in sharing information to better deal with 
the underground economy. 


2.8 The Department can improve its targeting of audits for the detection and reassessment of unreported 
income. As well, legislative opportunities exist to strengthen existing incentives to deter participation in the 
underground economy. 


Revenue Canada has agreed to take action to address our recommendations. 
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Statistics Canada 


Managing the Quality of Statistics 


Chapter 3 - Main Points 


3.1 Statistics Canada is committed to producing statistics of high quality. It has put in place a wide range of 
systems and practices to build quality into its statistical programs and to maintain an environment that encourages 
a concern for quality throughout the organization. However, the quality of the statistics it produces needs to be 
better documented and reported both within and outside the Agency. The Agency needs to integrate its many 
quality-related systems and practices better and adopt a more disciplined approach to documentation. 


3.2 The Agency has employed a number of formal quality assessment mechanisms, but individual programs 
have not applied them consistently. We concluded that the mechanisms currently used do not, either individually 

or collectively, provide systematic, transparent information on the adequacy of quality management systems and 

practices in the Agency’s statistical programs or on the quality that they actually achieve. 


Background and other observations 


3h8: Statistics Canada is responsible for collecting, compiling, analyzing and publishing statistical information 
on the economic, social and general conditions of Canada and Canadians. The statistics that the Agency produces 
support the development, implementation and evaluation of policies, programs and decision making in all sectors. 
They help us make informed decisions about such matters as where to live, what careers to pursue and how to 
vote. The Agency is widely respected among its peers, and has an international reputation for independence, 
innovation and quality that 1s second to none. 


3.4 Rapid social, economic and other changes have heightened the demand for reliable, objective statistical 
information on a wide variety of issues. As the demand for and use of statistics grow, their quality becomes 
increasingly important. Statistics Canada has identified six characteristics that its systems and practices for 
managing quality need to address: relevance, accuracy, timeliness, accessibility, interpretability and coherence. 


a3 The Agency carried out self-assessments of four major surveys for our audit. Each assessment reached a 
positive conclusion about the overall adequacy of quality management. The work was well planned and executed. 
We concluded that in three of the four surveys, the self-assessments provide reasonable assurance that quality 
management systems and practices are adequate. In the remaining case, however, we concluded that the 
weaknesses identified and the recommendations made are more important than the self-assessment suggests, and 
deserve the attention of senior management. 


3.6 While its policy on informing users of data quality and methodology is clear and well structured, the 
Agency’s implementation of the policy is inconsistent. Consequently, users are not always appropriately informed 
of the strengths and limitations of statistics. We also noted that while the quality of statistics figures prominently 
in its commitments to Parliament for results, the Agency’s most recent Performance Report, tabled in October 
1998, provides only limited information on the quality of the statistics that it produced. 


Statistics Canada’s responses to our recommendations are included in this chapter. The Agency either agrees 
to take action or notes that initiatives are under way in the case of six of the eight recommendations we 


make. In the remaining two cases, the Agency agrees with the intent of our recommendations and indicates 
that it will consider the issues further. 


8 Report of the Auditor General of Canada — April 1999 


Fisheries and Oceans 


Managing Atlantic Shellfish in a 
Sustainable Manner 


Chapter 4 - Main Points 


4.1 In October 1997, we reported on problems associated with the Department’s management of the Atlantic 
groundfish fisheries. In the current audit, we found that many of these problems also exist in the Department’s 
management of the Atlantic shellfish fisheries. For example, we noted increases in harvesting capacity and the 
encouragement of increased fisher participation through open access licensing in the shellfish fisheries. In 
addition, we found weaknesses in the information used in making resource decisions, and gaps in monitoring, 
control and surveillance. The full impact of these problems is not obvious, as most shellfish fisheries are currently 
recording high landed values. However, in our view these are significant concerns that must be addressed to 
ensure that the shellfish fisheries are managed in a sustainable manner. 


4.2 The Department’s decisions have a profound impact on those engaged in the fishing industry and the 
communities that rely on the income generated from the industry. The absence of a fisheries policy that fully 
reflects sustainability concepts means that decisions are made on an ad hoc and inconsistent basis rather than as 
part of an overall framework for achieving a sustainable fishery. An open and transparent process in which clearly 
articulated and consistently applied principles guide decision making would provide all stakeholders with 
assurance that their interests are considered and that the resource is protected over the long term. 


4.3 We observed resource use decisions that are not consistent with the Department’s currently stated 
objectives for fisheries management. As we reported in October 1997, there is a need to have the government 
clarify fisheries objectives in legislation. The Department needs to move forward with the development of a 
sustainable fisheries framework that incorporates the interdependent factors — biological, economic and social — 
that affect the fishery. 


Background and other observations 


4.4 In the 1990s, Atlantic Canada saw a virtual collapse of its commercial groundfish fishery (cod, haddock, 
pollock, halibut and various flatfish). In the same period, however, there was a general rise in the value of 
shellfish landings (lobster, scallop, snow crab and shrimp). In 1997, the landed value of all shellfish in Atlantic 
Canada was $920 million, which represented 81 percent of the landed value of all fish landed in the region. 


4.5 The Department has stated in its key parliamentary accountability documents that its objective is 
conservation, or protecting the productive capacity of the natural resource that supports the fishery. It has also 
reported that it has an economic objective, but the expected results for this objective are not stated. The 
Department has indicated that it is not responsible or accountable for social outcomes. We found that most 
resource use decisions in the shellfish fisheries are heavily influenced by social and economic factors. 


4.6 The Department’s “Fishery of the Future” strategy reflects objectives that include ensuring economically 
viable and self-reliant fisheries, over time. However, these objectives are not fully reflected in the Department’s 
reporting to Parliament. We found resource use decisions in the shellfish fisheries that are inconsistent with the 
concept of an economically viable industry. 


4.7 Co-management, designed to increase industry’s role, responsibility and accountability in fisheries 
management, is an important aspect of the Department’s Fishery of the Future strategy. Participants assuming 
greater responsibility for their industry is an important element of sustainability. However, very little power 
sharing has actually occurred. In our opinion, there are weaknesses in the Department’s current approach to 
co-management. 
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4.8 The Department has recognized that there are weaknesses in the fisheries management framework in the 
existing Fisheries Act. However, amendments to deal with these weaknesses have not yet been re-introduced in 
the House of Commons. 


Fisheries and Oceans’ responses to our recommendations are included in this chapter. The Department 
either agrees to take action or notes that initiatives are under way to address three of our four 
recommendations. The Department has not indicated an intention to take action at this time on our 
recommendation that addresses co-management. 
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Collaborative Arrangements 


Issues for the Federal Government 


Chapter 5 — Main Points 


5.1 Collaborative arrangements are an alternative way — a potentially more innovative, cost-effective and 
efficient way — to deliver programs and services that traditionally have been provided by federal government 
departments and Crown corporations. In collaborative arrangements, the federal government, other levels of 
government and organizations in the private and voluntary sectors agree to share power and authority in decisions 
on program and service delivery. 


5:2 With the growing use of these arrangements, more taxpayer dollars are being spent and the risks need 
correspondingly more attention. The risks include arrangements set up poorly among the partners, limiting their 
chances for success; partners not meeting commitments; insufficient attention to protecting the public interest; 
insufficient transparency; and inadequate accountability. 


FS We believe that serving the public interest, effective accountability and greater transparency are basic 
elements of a framework for these arrangements, and we suggest questions that parliamentarians might wish to 
raise when assessing them. 


Background and other observations 


5.4 In the desire for greater efficiency, it is very important that the federal government and its partners not 
lose sight of the public purpose behind the collaborative arrangement, and of the need to provide transparent, fair 
and equitable service to the public. 


55 Effective accountability is more complex in a collaborative arrangement. The federal government is 
accountable to Parliament for the use of federal funds and authorities, to its partners for keeping its commitments, 
and, with its partners, to the public for the results the arrangement produces. In our view, this shared 
accountability means that more parties are accountable and it in no way lessens the federal government’s 
accountability for its own responsibilities 1n the arrangement. 


5.6 Delivering programs and services to the public through collaborative arrangements often requires more 
transparency than traditional delivery by a government department. Because partnerships are involved, it may be 
more difficult for citizens to know who is responsible. Consequently, the federal government needs to be as open 
as possible with information about agreements, decisions and results of the arrangements. 


Se], The Treasury Board Secretariat has informed us that it intends to continue to provide advice and to 
develop guidance on collaborative arrangements for federal departments and agencies that will address many of 
the issues identified in this study. 
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Human Resources Development 
Canada — Accountability for 
Shared Social Programs 


National Child Benefit and 
Employability Assistance for People 
with Disabilities 


Chapter 6 - Main Points 


National Child Benefit 


6.1 The National Child Benefit (NCB) represents a new form of collaborative arrangement between 
provinces (except Quebec) and territories and the federal government. A key challenge for all involved is to assure 
taxpayers that moneys are spent for the purposes intended, with due regard to economy and efficiency and with 
appropriate means to measure and report on effectiveness. Because there are many governments involved, it is 
also necessary to respect the jurisdictional competence of the different parties. As the NCB is implemented it is 
critical that, at a minimum, there be no less accountability because it is shared than if only one jurisdiction were 
involved. 


Background and other observations 


6.2 The goal of the National Child Benefit is to reduce the depth of poverty among families with children and 
to increase parental attachment to the work force. The NCB involves no new law, contract, or contribution 
agreement, but rather an increased federal child tax benefit (the NCB supplement) for low-income families — 
about $850 million in 1998, rising to $1.7 billion by 2000. In turn, jurisdictions providing social assistance 
benefits to families may reduce their payments by the amount of the increased tax benefit. They have agreed to 
reinvest these savings in programs that have mutually agreed-upon objectives and that benefit poor families with 
children. 


6.3 One of the clear achievements of the negotiations leading to the NCB was the partners’ agreement on the 
overall goals of both the federal and the provincial elements of the program. They also have committed to a new 
kind of joint accountability to the public. What is distinctive about this commitment is that no level of government 
is more responsible for reporting on the results than another. Each is accountable for the overall program. 


6.4 It will be a significant challenge in the first few years, before evaluation results are available, to 
demonstrate precisely how the National Child Benefit has contributed to reduced depth of poverty and increased 
employment among its recipients. In assessing progress, the NCB accountability report will also need to be clear 


about any trade-offs among goals and to discuss the implications. Otherwise, readers may expect that all goals can 
be achieved at the same time. 


6.5 There are potential gaps in the quality of financial and other information. The credibility of the 
accountability reports depends on the comparability, accuracy and verifiability of information from all parties — 
not only those who negotiated the arrangement (provinces, territories and the federal government) but also those 
who are responsible for designing and implementing only specific sub-programs (Ontario municipalities and First 
Nations). This is also a challenge for audit offices that serve the partners in this arrangement. 


6.6 We think the key ingredients in reporting the NCB’s results to the public are transparency and adequacy 
of the information. This means: 
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¢ appropriate care is taken to ensure that the information is credible and, at least for financial 
information, subject to audit; 


e = data are adequate to determine if the program’s overall goals are being achieved; and 
e data are sufficiently comparable that outcomes of different provincial approaches can be compared. 


It must also be clear whose responsibility it is to do all this, and who will provide assurance that it has been done. 
Moreover, those charged with the responsibility must have the capacity to undertake the task. 


The Department has said that this chapter will be of considerable assistance in its work to build an 
accountability regime for the NCB. Some issues are already being addressed. The remaining issues identified 
in this study will be raised by the federal government in its capacity as co-chair of the federal-provincial- 
territorial group working on the NCB. 


Employability Assistance for People with Disabilities 


6.7 Employability Assistance for People with Disabilities (EAPD) is the other program to emerge from 
federal-provincial discussions in 1997—98 on the social union. The aim of this 50—SO cost-shared arrangement is to 
help people with disabilities overcome the barriers they face in the labour force. EAPD supports provincial and 
territorial programs and services ranging from programs that assist in the first steps toward developing skills to 
those that support a person at work so the person can keep working. Our interest in the program at this early 
juncture is that steps be taken to ensure that the information each partner will provide on its own expenditures and 
programs is credible, and permits comparisons of different approaches and assessments of the overall program’s 
effectiveness. 


Background and other observations 


6.8 Each province has signed an agreement with the federal government. Taken together, the agreements 
illustrate several of the elements of a collaborative arrangement discussed in Chapter 5 of this Report. For 
instance, the governments agree to follow a co-ordinated, participative planning process and to evaluate program 
results. 


6.9 Human Resources Development Canada (HRDC) and provincial partners have set about to solve 
implementation issues in a collaborative fashion. For example, the provinces have committed to annual 
accountability to HRDC. However, so far there is no specific commitment to the format, substance, or timing of 
any overall annual report, nor are there specific goals or targets over any time frame. We recommend that HRDC 
prepare a brief overall annual report for EAPD, comparing activities, expenditures, program outputs and 
performance of the partners. 


6.10 The partners still have important work to do. When so many jurisdictions are involved, those with 
oversight and audit responsibilities face the challenge of helping to ensure that the quality of financial and 
performance reporting is maintained and enhanced. 


The Department has said that the case study will be useful in its work with provinces. It agrees in principle 
with the recommendation to produce an overall annual report on EAPD expenditures and performance. It 
pointed out that it will be important to note that EAPD is only one part of overall programming for people 
with disabilities. 
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The Atlantic Groundfish Strategy 


Contributions and Grants 


Chapter 7 - Main Points 


Tel We have little assurance that all contributions under The Atlantic Groundfish Strategy (TAGS) were used 
for their intended purposes. These were part of TAGS active labour adjustment measures managed by Human 
Resources Development Canada (HRDC). 


Ted Most of the deficiencies noted in the audit relate to a lack of diligence in assessing project proposals and 
signing contribution agreements, as well as lack of monitoring by the Department. 


re) Many of the files contained no project proposals; in others, proposals were not sufficiently developed to 
allow proper assessment. Some agreements lacked complete information, included ineligible costs, or did not 
correspond to the measure under which the project was funded. There was little evidence of on-site monitoring 
visits to examine expense records. 


Background and other observations 


7.4 The Atlantic Groundfish Strategy, in effect from 16 May 1994 to 29 August 1998, comprised measures to 
assist those affected by the groundfish moratorium. About $150 million of the $1.9 billion allocated to TAGS was 

spent on active labour adjustment measures such as training, mobility assistance, wage subsidies and employment 

bonuses. 


iPS Those expenditures were not covered in our audit of The Atlantic Groundfish Strategy, reported in 
Chapter 16 of our October 1997 Report. Following the publication of that Report, the House of Commons 
Standing Committee on Fisheries and Oceans requested that we audit the expenditures. 


7.6 Our Office and the Internal Audit Bureau (IAB) of Human Resources Development Canada agreed that 
[AB would audit the grants and contributions made under TAGS. We closely monitored and reviewed that audit so 
we could rely on the findings for this chapter. 


Tel TAGS had increased the caseload of officers by some 40,000 participants over a short period of time. 
This created pressure to identify, approve and contract for an unprecedented number of projects. 


7.8 Several contributions went to projects that were not clearly related to TAGS. More than half of the 
agreements were signed after projects had begun. Expenditures were reimbursed without supporting 
documentation for the claims. For more than half of the projects reviewed, there was no evidence that the files had 
been closed, although most of those projects had been completed more than two years earlier. 


Overall, Human Resources Development Canada agrees with the findings of the audit and recognizes that a 
number of important points and concerns are raised in the chapter. Actions undertaken to address the issues 
include the development of new policies and procedures, training for managers and staff and, in new 
initiatives, securing resources to ensure sufficient monitoring. 
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The Atlantic Groundfish Strategy 


Follow-up 


Chapter 8 — Main Points 


8.1 We believe the government’s efforts to implement the recommendations in our October 1997 Report 
Chapter 16 on The Atlantic Groundfish Strategy (TAGS) have been satisfactory. 


8.2 New fishery restructuring and adjustment measures are being implemented. In contrast to our 
observations of 1997, the eligibility criteria for the new measures are clear, logical and applicable. The 
accountability framework established for the measures corrects the shortcomings we had identified in TAGS: it 
clearly defines the responsibilities of the organizations involved, sets out an overall strategic plan and provides for 
a formal co-ordination mechanism. 


Background and other observations 


8.3 The objective of TAGS (1994-98) was to restructure the fishery industry in Atlantic Canada to make it 
economically viable and environmentally sustainable. 


8.4 Close to $1.9 billion was allocated to TAGS. Most of the funding was used to provide income support to 
the some 40,000 fishers and plant workers affected by the groundfish moratorium. 


8.5 In October 1997, we urged the government to carefully examine the impact of the decisions made under 
TAGS in order to benefit from the valuable lessons that could be learned. 


8.6 This follow-up examined progress to date in addressing our 1997 recommendations. Some of our 
observations in this chapter relate to TAGS and others to the Atlantic fishery restructuring and adjustment 
measures announced in June 1998. 


8.7 Human Resources Development Canada completed an evaluation of the labour adjustment component of 
TAGS. It also reviewed the duration of eligibility of TAGS participants. The departments involved in 
implementing the new measures now have considerable information on the profile of the targeted populations, in 
sharp contrast to the situation that prevailed when TAGS was developed. 


8.8 We believe the orderly way in which the departments are implementing the new fishery restructuring and 
adjustment measures represents satisfactory progress. 
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Management of Science and 
Technology Personnel 


Follow-up 


Chapter 9 - Main Points 


D1 We are satisfied with the efforts made by the science and technology community to follow up on our 
audit recommendations of 1994 and the concerns we raised in our 1996 follow-up report. In our view, the 
community is showing leadership and perseverance in dealing with the human resource management issues we 
raised. 


oe As aresult of all the work done since 1994, the science and technology community is now in a position to 
act. But the community is faced with considerable challenges. It must give priority to resolving the oncoming 
changes in its demographic profile that could weaken or compromise the government’s science and technology 
capacity. It must tackle the dual challenges of attracting and recruiting promising young scientists and 
technologists while retaining high-calibre employees to mentor and develop the new recruits. The community is 
thus seeking new measures, tools and resources for external recruitment, as it estimates that over the next five 
years it may have to recruit between 2,500 and 3,300 employees to build a renewed and rejuvenated science and 
technology work force. 


Background and other observations 


953 At 31 March 1998, the federal government had close to 20,000 scientific and technical employees 
working in science-based departments, agencies, Crown corporations and research establishments in the fields of 
natural sciences and engineering (hereinafter referred to as the “science and technology community”). 


9.4 The science and technology community makes an important contribution to the government’s 1996 
science and technology strategy, which focusses on sustainable job creation and economic growth, improved 
quality of life and the advancement of knowledge. 


a Expenditure and work force reduction in the public service has changed the profile of the science and 
technology community and worsened the long-identified problems of rejuvenation and recruitment. The change in 
the age profile is a major challenge to the future of the community. Not only have most senior and experienced 
scientists and technologists left the government since 1994, but the youngest and most promising as well. 


9.6 Following our 1994 audit of federal science and technology activities, the community mobilized to 
develop a management framework and a results-oriented plan for human resources management in science and 
technology. It addressed such issues as the need for a more strategic approach to the management of scientific 
personnel; for more systematic renewal of scientific personnel; and for more effort to maintain the skills and 
knowledge base in research establishments. Working groups were created to study important human resource 
issues. Among their recommendations to the Science and Technology Senior Steering Committee on Human 
Resources was that new mechanisms be adopted and human resource strategies developed to improve the 
management of science and technology personnel in science-based departments and agencies. 
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D7 In 1994, we pointed out the need to develop a stronger and more effective management capability. Since 
then, the community has developed a competency profile for science and technology managers. During our 
consultations, we noted that most science-based departments used their own competency profiles instead of the 
one developed by the working group. Moreover, their profiles were being used solely to identify training needs 
and generally not for purposes of manager recruitment, promotion or performance assessment. Present practices 
suggest a lack of consensus in the community on the management competency profile defined by the working 
group. This could eventually prevent the integration of recruitment and training activities as well as the reward, 
promotion and compensation systems envisioned in the Science and Technology Blueprint for Human Resources 
Management. 


9.8 The Treasury Board Secretariat, science-based departments and agencies and the science and technology 
community have indicated that they are committed to following through on the strategies and plans developed to 
date. 
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Indian and Northern Affairs 
Canada 


Funding Arrangements for First 
Nations: Follow-up 


Chapter 10 - Main Points 


10.1 Indian and Northern Affairs Canada still needs to better match the level of flexibility attached to funding 
arrangements with the willingness and ability of First Nations to assume responsibility for billions of dollars they 
receive annually through the Department. 


10.2 The Department is not taking adequate steps to ensure that allegations of wrongdoing, including 
complaints and disputes related to funding arrangements, are appropriately resolved. Redress (resolution 
mechanisms) needs to be improved as an element of accountability. 


10.3 The Department has stated that it has slowed the rate at which the Financial Transfer Arrangement (FTA), 
a new type of funding arrangement, is being implemented in order to address issues concerning the willingness 
and ability of First Nations to adopt it. The Department still has a long way to go if it wishes to achieve its 
objective of implementing the FTA as the appropriate funding mechanism to replace other types of funding 
arrangements. It will need to find ways to expedite the conversion process while improving co-ordination of 
funding with other federal departments. 


Background and other observations 


10.4 Funding arrangements are a key element in the relationship between First Nations and the federal 
government. Parliament appropriates about $4 billion annually to Indian and Northern Affairs Canada to fund 
several programs for First Nations communities. The programs include social assistance, education, capital 
facilities, housing, and economic development for approximately 600 First Nations and other Aboriginal groups. 


10.5 In our November 1996 Report, we made recommendations relating to funding arrangements, including 
the FTA, and to the co-ordination of funding across federal departments. We also made recommendations on 


accountability issues and on the suitability of funding arrangements for the needs of the Department and First 
Nations. 


10.6 This follow-up focusses on the Department’s implementation of our 1996 recommendations. 


The Department believes that with respect to redress, the follow-up extends beyond the issues raised in 1996; 


however, it has acknowledged that continuing effort is needed to improve certain aspects of funding 
arrangements, including accountability. 
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Agriculture Portfolio 


User Charges 


Chapter 11 - Main Points 


11.1 We noted a number of good practices in the way Agriculture and Agri-Food Canada, the Canadian Grain 
Commission and the Canadian Food Inspection Agency have designed and managed fees they charge to users of 
their services. But these organizations do not yet meet all the requirements of the government’s policy on user 
charges. Although in recent years they have made progress, they are losing momentum and user charges are not 
being updated as programs change. This could result in some user groups paying either more or less than their fair 
share. 


12 Our audit found that the organizations often view user fees primarily as a means of generating revenue, 
ignoring their potential to achieve other possible benefits. For example, the Canadian Food Inspection Agency 
often charges the same fee for inspections whether or not food inspectors have found deficiencies, even though an 
inspection that has found problems entails additional time. This provides no incentive for the industry to comply 
with regulations and thereby minimize the time government inspectors need to spend. 


11.3 The organizations we audited need to make major improvements in the way they report to the public, the 
industry and Parliament on their plans for user charges and on the results. Despite the fact that this is a contentious 
issue that has sparked a great deal of interest from Parliament, the organizations in most cases have not publicly 
explained why specific user charges were needed and how they would be applied. Nor have their reports provided 
enough information to allow readers to judge whether user charges are managed properly. 


11.4 The three organizations need to improve their management of user charges in a number of areas, 
including costing their services, assessing the impact of fees, establishing formal appeal processes and integrating 
user charges into their strategic planning. 


Background and other observations 


BS In its 1997 Cost Recovery and Charging Policy, the federal government states that user charges should be 
implemented for “services that provide identifiable recipients with direct benefits beyond those enjoyed by the 
general public, unless overriding policy objectives would be compromised.” A key underlying principle is that 
user charges provide benefits much broader than the revenue generated, such as making services more responsive 
to users’ needs. 


11.6 Organizations in the agriculture portfolio charge for many different services. For example, the Canadian 
Grain Commission charges the grain industry for weighing and inspecting grain destined for export. The Canadian 
Food Inspection Agency charges importers and exporters to inspect plants and plant products for pests or diseases. 
Agriculture and Agri-Food Canada charges owners of cattle that graze on community pastures for services it 
provides. 


|B User charges have been a focal point of much concern in the agriculture sector. Over the past five years, 
revenues from user charges in certain areas have increased by as much as 300 percent. Producers and the industry 
have objected to the way the fees were introduced and to the cumulative impact they have had. Some have 
contended that the fees were simply a tax. 


11.8 A moratorium on new fee increases was announced at the time the Canadian Food Inspection Agency 
was created, to be in effect until the year 2000. Officials of the three organizations have given many different 
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interpretations of not only how long the moratorium will last but whether it applies to other organizations in the 
portfolio. We note that the organizations we audited believe they are constrained from correcting even known 
inconsistencies among fees and responding to changes in their business environment. In our view, the period of the 
moratorium offers them an opportunity to reassess their current position, develop guiding principles and plan 
improvements. 


11.9 The organizations face specific challenges in that they recover fees from individuals they also regulate. It 
can be difficult to establish the amount that users of a service should be charged for the benefits they receive when 
the service also benefits the general public. There is a need for broader consultation, where possible, that takes 
into account the interests of both industry and the public. Those who pay fees want to influence how services are 
provided, and the organizations must ensure that the needs of all stakeholders are taken into account. 


11.10 The three organizations need to make key decisions on user charges and take action to resolve the 
following situations: 


e Recent changes in its operations and in the grain industry have made the existing fee structure of the 
Canadian Grain Commission no longer appropriate. 


e The Canadian Food Inspection Agency initiated fee increases but has not completed its plan for user 
charges. Consequently, commodity groups are paying different proportions of the Agency’s costs for 
similar service, which represents inconsistent treatment. 


e Agriculture and Agri-Food Canada has not explained adequately when, how and why user charges 
are implemented for each of its programs. Consequently, the reasons why some programs have user 
charges and other, similar programs do not remain unclear. 


The responses to our recommendations from Agriculture and Agri-Food Canada, the Canadian Food 
Inspection Agency and the Canadian Grain Commission are included at the end of this chapter. Each of the 
organizations has said that it will act upon the recommendations that apply to it. 
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Agriculture and Agri-Food Canada 


A New Crop: Intellectual Property 
in Research 


Chapter 12 - Main Points 


12.1 The context of intellectual property related to food production in Canada has changed dramatically in the 
last 10 to 15 years. This is forcing Agriculture and Agri-Food Canada’s Research Branch to seriously rethink its 
way of managing its own intellectual property. It faces the following issues: 


e What should the Research Branch do when it makes a discovery that leads to an invention? Should it 
publish it, or legally protect it and then license it to others? If it does license it, should it license it to 
one organization or many? 


e How should the Research Branch deal with intellectual property of others that it uses in its own 
research? For example, if it develops a new potato by taking advantage of private sector patents, will 
this hinder its ability to get the new product into the hands of producers? Should the Branch buy 
access to private sector patents? Should it barter with its own patents? 


We believe that it is important for the Branch to answer these questions — how best to release its intellectual 
property and how best to acquire the intellectual property of others — because they have implications for the 
future success of agriculture. As Canada’s agriculture sector becomes more and more sophisticated, it depends 
increasingly on exchange of and access to new knowledge coming from the Research Branch and other 
organizations. Answers to these questions also have implications for future government revenues from intellectual 
property. 


12.2 Particularly urgent is the risk to the Branch’s continued ability to provide the agriculture sector with 
innovations. The risk stems from its significant use in research projects of intellectual property owned by others. 
Should it encounter significant obstacles to procuring the rights to use these proprietary technologies, the Branch 
may have to alter, postpone or abandon current research initiatives. 


25 The Research Branch is endeavouring to cope with these challenges, but this has become difficult due to 
substantial changes in the Branch and in the agri-food sector. In our opinion, the Branch needs to hasten and 
expand its efforts. A decision framework is needed to guide employees in deciding which research products should 
be legally protected, how to best acquire new intellectual property and how to best release its own protected 
intellectual property. Other tasks facing the Branch are to improve communication with staff; improve information 
on the Branch’s intellectual property; and improve the integration of stakeholders’ views into decision making. An 
overall approach to managing intellectual property is needed. 


Background and other observations 


12.4 Intellectual property is information that is useful and transferable and controlled by someone. It includes 
inventions, some of which are controlled through patents and others that are controlled by secrecy. It also includes 
plant varieties and plant materials. Until recently, there were few ways to protect intellectual property stemming 
from agricultural research, and new crops offered little monetary return to the breeder. New crop types can now be 
protected under the 1990 Plant Breeders’ Rights Act, and some new biotechnology processes can be protected by 
patent. 


Report of the Auditor General of Canada — September 1999 


125 The Research Branch is the part of the federal agriculture department that does scientific research. It has 
a budget of about $250 million annually and is a key player in agricultural research in Canada. It is a major 
contributor to “Innovating for a Sustainable Future”, one of the Department’s business lines. It first made its mark 
more that 100 years ago with the release of Marquis wheat, which greatly helped develop Prairie agriculture. 
Current revenue from the Branch’s intellectual property licenses is around $3.4 million a year, a minor but 
growing part of its $250 million budget. 


12.6 The challenges the Branch faces in managing intellectual property include the accelerated pace of 
change, especially in the field of biotechnology; uncertainty in the Branch brought on by downsizing and a greater 
focus on research partnerships; the need to consult broadly on intellectual property policy; the high cost of 
protecting and enforcing its intellectual property rights; the need to understand the market for its intellectual 
property; and the need for good management information. 


The Department has accepted our recommendation and has committed its Research Branch to work to 
develop and communicate a general framework for managing the Branch’s intellectual property. 


23) 
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National Defence 


Hazardous Materials: Managing 
Risks to Employees and the 
Environment 


Chapter 13 - Main Points 


ne Our audit at 10 Canadian Forces bases found widespread, frequent and recurring instances in which 
National Defence did not meet the legal and policy requirements that govern hazardous materials. Such instances 
may put employees and the environment at increased risk. 


13.2 The Department has made efforts to improve its management of hazardous materials. While there are few 
clear measures of the impact of these efforts, we did find that injury rates for hazardous material injuries that 
require at least one day off work declined by more than half from 1993 to 1997. 


13:33 The 10 bases that we audited do not periodically monitor their air or liquid effluent emissions for all 
hazardous material contaminants on which limits are set by federal laws or guidelines. As a federal agency, the 
Department is not legally bound by provincial or municipal laws and bases do not monitor emissions and effluents 
against all of those standards. 


13.4 There is a lack of information on the overall state of compliance with legal and policy requirements. The 
existing management systems to ensure compliance at the base and unit levels do not promote continual 
improvement. Objectives are not set, plans do not exist, audit and inspection programs are not consistent and 
performance information is lacking. These are major factors in the instances of non-compliance by the 
Department with legal and policy requirements. 


Background and other observations 


ISS National Defence uses more than 6,000 hazardous products, such as gasoline, ammunition and battery 
acid. The Department estimates that at least one quarter of its 80,700 full-time employees come into frequent 
contact with hazardous chemical products, and that every worker may have some contact with hazardous 
materials. 


13.6 It is the Department’s policy that its hazardous materials management activities must meet or exceed the 
letter and spirit of applicable federal acts, regulations, policies and guidelines and, where appropriate, be 
compatible with provincial acts and municipal and international standards. The Department has not defined the 
meaning of “where appropriate”. 


13.7 Our audit focussed on how the Department is implementing its policy and programs related to hazardous 
materials. We looked at policies, procedures and results at National Defence headquarters, and conducted field 
work at 10 bases. We also reviewed how the Department is implementing relevant sections of its sustainable 
development strategy. 


13.8 Leadership is needed to set targets, to identify and monitor performance measures, and to follow up to 
ensure that programs are implemented. Commanders and managers have opportunities to demonstrate leadership 
when they revise the Department’s sustainable development strategy and implement environmental management 
systems. 


The Department responded positively to all our recommendations and proposes to take action that should 
address the problems our audit identified. In particular, the Department told us it is committed to rectifying 
situations of non-compliance with the requirements of the Workplace Hazardous Materials Information 


to 
tn 
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System; reviewing its effluent monitoring procedures and plans for air emissions; identifying provincial and 
municipal regulations and policies applicable at each base and wing; and establishing core requirements for 
audit and inspections to ensure consistency. 


The Department did not fully accept our conclusions about the level of non-compliance, contending that its 
audits have identified relatively few instances each year. 


* 
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National Health Surveillance 


Diseases and Injuries 


Chapter 14 - Main Points 


14.1 We observed weaknesses in national surveillance of diseases and injuries that, taken together, have clear 
national implications for public health. First, they compromise Health Canada’s ability to detect, anticipate, 
prevent and control health risks associated with outbreaks of communicable diseases and with other health threats. 
Second, they compromise its ability to plan, carry out and evaluate public health programs and other programs 
that deal with the causes and treatment of diseases. 


Background and other observations 


14.2 Health surveillance is a core function of public health. National surveillance of communicable and 
chronic diseases and of injuries is conducted mainly by Health Canada’s Laboratory Centre for Disease Control 
(LCDC), part of the Health Protection Branch. LCDC depends greatly on its interaction and collaboration with the 
provinces and territories and a variety of other federal departments and non-governmental organizations. 


14.3 There is no specific legislation, policy or agreement that links separate components of public health 
functions at the various levels of government. Indeed, there is a void: current health surveillance activities are 
largely carried out on an ad hoc basis. 


14.4 We found few formal agreements or protocols in place to prevent the entry into Canada of serious 
communicable diseases and to deal with disease outbreaks and threats to public health. The lack of attention to 
formalizing the way these threats are to be managed places the health of Canadians at undue risk. 


14.5 Key surveillance systems that we looked at were not working as intended. For a number of reasons, they 
were not enabling Health Canada to effectively monitor communicable diseases such as influenza, AIDS, 
tuberculosis and food-borne disease. The same is true of systems for surveillance of injuries and chronic diseases 
such as cancer, diabetes, and heart disease and stroke. We also found that performance measurement and reporting 
of results need to be improved. 


14.6 Work has begun on strengthening the Health Protection Branch’s future surveillance capacity. An 
important step in this regard is the Branch’s support of an integrated national surveillance network for public 
health information. It is unclear when such a network will be implemented, and it is paramount that all parties 
involved in this network remain committed to its implementation. 


Health Canada’s responses to our recommendations are included in this chapter. The Department concurs 
with the recommendations and has agreed to take corrective action. In some cases, this action is already 
under way. 


Nw | 
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LLP SL a 


Management of a Food-Borne 
Disease Outbreak 


Chapter 15 — Main Points 


15.1 In the spring of 1998, there was a nation-wide outbreak of a food-borne disease; it was one of the largest 
outbreaks of food-borne disease in Canadian history and involved the investigation of more than 800 reported 
cases across Canada. Over 80 percent of the affected were children under 15 years of age. At least 60 were 
hospitalized. 


15-2 Some important aspects of the response to the outbreak worked well, but others did not. The 
contaminated product was identified quickly and its removal from points of sale was started immediately after the 
issuing of a recall. However, there was a lack of timely exchange of information to identify the scope of the 
outbreak. There was also a lack of full co-operation among the agencies involved in the response to this outbreak. 
The Canadian Food Inspection Agency (CFIA) did not share certain distribution information when requested by 
provincial public health officials to assist in the investigation. In addition, the CFIA’s abrupt decision not to lead 
one of the plant inspections resulted in confusion and unnecessary delay. 


15:3 A formal framework is needed that sets out clearly the roles and responsibilities of Health Canada’s 
Laboratory Centre for Disease Control (LCDC) in relation to those of other participants, in order to guide the 
response to threats to public health. Many individuals could have avoided this illness had the federal and 
provincial health departments acted more quickly. 


Background and other observations 


15.4 The case described in this chapter illustrates many of the issues discussed in Chapter 14 on National 
Health Surveillance and the management of outbreaks and threats to public health. The audit looked at how 
federal and provincial agencies — Health Canada’s Laboratory Centre for Disease Control and Food Directorate, 
the Canadian Food Inspection Agency, and provincial and local public health departments — responded to this 
nation-wide outbreak of a food-borne disease. 


15.5 Food-borne diseases have important implications because of the wide distribution of food products and 
the resulting potential for affecting very large numbers of people spread over wide geographic areas. 


15.6 The activities of the Laboratory Centre for Disease Control involve the timely investigation and control 
of disease outbreaks, often in collaboration with provinces and other federal agencies — in particular, with the 
Canadian Food Inspection Agency in the event of a disease caused by food. The CFIA is responsible for 
enforcement actions in food-related emergencies and is to take a lead role in investigations and co-ordination of 
food safety emergency responses. Health Canada’s Food Directorate is responsible for assessing the effectiveness 
of the CFIA’s food safety activities. Provincial and local medical officers of health have a legislative mandate to 
investigate disease outbreaks, and provincial laboratories provide laboratory services. 


18:7 We found that LCDC was not well prepared to manage disease outbreaks. It had no established operating 
procedures to respond to food-borne disease outbreaks. In addition, there were no formal protocols between 
Health Canada, the CFIA and the provinces that clearly defined procedures and the role of LCDC in relation to the 
roles of other participants in the investigation of disease outbreaks. 
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15.8 There was a lack of transparency in the post-outbreak reviews undertaken by LCDC and by the CFIA. In 
such situations, particularly given that the cause of the contamination was never found, we believe it is in the 
interest of public health that all participants contribute to, and learn from, such reviews. 


Responses to our recommendations from Health Canada and the Canadian Food Inspection Agency are 
included in this chapter. Both Health Canada and the Canadian Food Inspection Agency concur with the 
recommendations and have agreed to take corrective action. In some cases, this action is already under way. 
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REET TEATS 


Revenue Canada 


Goods and Services Tax: 
Returns Processing and Audit 


Chapter 16 — Main Points 


16.1 We found that Revenue Canada’s Summerside Tax Centre, until recently its sole GST processing centre, 
has steadily become more efficient. It has streamlined several processes, enhanced its error correction procedures, 
reduced unit costs and improved the timeliness of its processing operations. As a prelude to decentralizing the 
processing of GST returns, Revenue Canada further simplified its processing procedures. The results of a pilot test 
point to further efficiency gains. 


16.2 At present, standards for returns processing tend to place insufficient emphasis on quality, accuracy, 
timeliness and unit cost, and the use of performance information is hampered by problems with the availability 
and reliability of data. Better performance measures would help to manage interest costs and improve service to 
registrants. 


16.3 Revenue Canada can do more to improve the performance of its set of automated validity checks. These 
checks are the cornerstone of its pre-payment audit program. However, they are not particularly discriminating in 
their initial targeting of inappropriate refund claims. Revenue Canada also needs to consider reviewing and 
auditing returns on which refunds are not requested at the same time as it conducts pre-payment audits of refund 
requests. 


16.4 Many of Revenue Canada’s post-payment audits consume too many staff hours and go on for too long. 
Revenue Canada needs to help its auditors shorten their audit times by providing closer supervision and by 
encouraging better planning, execution and control of audit work. Audit performance may also be improved by 
better selection of registrants for audit. 


Background and other observations 


16.5 The introduction of the Goods and Services Tax (GST) in 1991 had a huge impact on Revenue Canada’s 
activities. The Department had to develop a completely new processing operation and techniques to verify the 
accuracy of the returns submitted by almost 2 million registrants, ranging from small entrepreneurs to 
multinational corporations. 


16.6 In 1993, Revenue Canada opened a new tax centre in Summerside, Prince Edward Island to process all 
GST returns and rebate forms from non-Quebec registrants. (The ministére du Revenu du Québec administers 
GST processing and audit of Quebec registrants.) The Summerside Tax Centre employs the equivalent of about 
660 employees and has an annual operating budget of about $28 million. 


16.7 Revenue Canada now processes about 5.5 million GST returns each year, as well as 200,000 domestic 
rebate forms and 3 million remittances from GST registrants. 


16.8 The Department has begun to decentralize a portion of its GST processing activities to tax centres across 
the country. The processing of rebate forms will continue to be done in Summerside. 


16.9 While, in general, GST returns processing is a tax centre activity, Revenue Canada’s audit programs are 
performed by auditors in tax services offices across the country. There are two major audit programs. The first is 
the pre-payment audit program, which accounts for about 25 percent of auditor time. After the data have been 
processed but before refunds are issued, Revenue Canada applies a variety of automated validity checks to ensure 
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that requests for refunds are appropriate. The checks are designed to capture the requests with the highest risks of 
non-compliance. The captured refund claims are sent electronically to tax services offices across Canada for 
review and, in some cases, for audit. In 1997-98, the GST pre-payment audit program resulted in assessments of 
about $165 million. 


16.10 Revenue Canada also performs post-payment audits. These audits account for about 75 percent of auditor 
time and focus not only on particular returns but also on registrants with the highest risk of non-compliance. In 
1997-98, this work resulted in assessments of about $355 million. 


16.11 We examined the efficiency and effectiveness of the GST returns processing and audit functions. The 
efficient and effective use of resources is important because it has a direct impact on protecting the public purse 
and improving service to registrants. 


Revenue Canada’s responses to our recommendations are included in this chapter. The Department is in 
agreement with the seven recommendations, and its responses describe a number of actions that have been 
completed or are in progress to deal with them. 
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Canada Infrastructure 
Works Program 


Phase II and Follow-up 
of Phase | Audit 


Chapter 17 - Main Points 


1D The follow-up to our 1996 audit chapter indicates limited progress in addressing the deficiencies we 
identified in Phase I of the Canada Infrastructure Works Program. Our 1996 observations dealt with the need for 
clearer definitions of program objectives and project selection criteria, as well as the need for improved 
information, including better measurement and reporting of job creation. The main area of improvement has been 
environmental assessments; as well, a start has been made in responding to the need for compliance audits. 
However, most of our concerns have not been addressed. 


17.2 The Program was implemented within very demanding time frames. Federal and provincial program 
managers view the program approach positively and regard it as having contributed to strong federal-provincial 
relations and co-operation. The Program’s output from 1994 to date has involved mobilizing and co-ordinating the 
efforts and resources of three levels of government and other partners to plan and implement more than 17,000 
engineering and construction projects nation-wide. 


17.3 Our audit of Phase II indicates that, from an overall federal perspective, the Program 1s essentially 
“running on trust” with little accountability. Criteria for project selection were not clearly defined, and many of 
the files we reviewed lacked persuasive evidence to justify applicants’ claims relating to selection criteria. In most 
cases, federal officials recommended projects for approval without ensuring that applicants’ claims were 
adequately supported. 


17.4 We found that federal officials relied on municipal and provincial certifications with respect to costs 
claimed. The implementation of compliance audits, which, among other things, represent a means of obtaining 
assurance on the adequacy of financial controls, has been slow. 


17.5 There are incentives for provincial governments to transfer their budgetary resources away from their 
own programs to the federal infrastructure initiative. However, this program substitution is not inevitable; in two 
provinces, for example, safeguards were put in place in Phase I to limit the “substitution effect”. Any substitution 
that does occur reduces the Program’s infrastructure development and job creation benefits. The Treasury Board 
Secretariat has not set out the limitations of estimates of employment generated by the Program in reporting them 
to Parliament. 


Background and other observations 


17.6 The Canada Infrastructure Works Program was introduced by the federal government in 1994 as a 

$6 billion temporary shared-cost initiative with the objectives of assisting in the maintenance and development of 
infrastructure in local communities and the creation of employment. In 1997, the government announced an 
extension (Phase II), involving an additional $425 million provided by the federal government, to be matched by a 
further $850 million from the provinces and other partners. For the most part, construction under both phases was 
scheduled to be completed by 31 March 1999. 


WES Construction of roads, bridges, and water and sewer networks predominated in both phases, accounting 


for well over 60 percent of total expenditures. Community, cultural and recreational services were of less 
significance in Phase II. 
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17.8 The Canada Infrastructure Works Program is a contributions program, with a requirement for the 
payment of federal funds to be subject to performance conditions being met (such as project targeting) and to 
compliance with program requirements. However, our examination of the project files indicated that program 
officials have not adequately addressed these concerns. 


17.9 Finally, we found that progress has been made under Phase II in conforming with the requirements of the 
Canadian Environmental Assessment Act. However, where negative environmental effects have been identified, 
the specification and monitoring of mitigation measures remain inadequate. 


The Treasury Board Secretariat indicated that although improvements can and should be made in any 
future such programs, the chapter presents an inappropriately negative view of what was a highly successful 
program in terms of results and one that made a positive contribution to federal-provincial-municipal 
relations in Canada. 
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Public Works and Government 
Services Canada 


Alternative Forms of Delivery: 
Contracting for Property 
Management Services 


Chapter 18 - Main Points 


18.1 The contracting process used by Public Works and Government Services Canada (PWGSC) to contract 
out the operation and maintenance of two thirds of its buildings was well managed and consistent with the 
objectives of the Department. We found that the bidding process was open, transparent and fair. The contracts 
were well designed and the handover to the contractor went smoothly. A significant feature of the contracting 
process was the use of “fairness monitors” to provide independent assurance to the Department’s senior 
management that the process was conducted fairly and in accordance with the rules. 


18.2 Because some of the key monitoring and control features were not in place at the time of the handover of 
operations, ongoing management of the contracts has been problematic. Important items that had to be ready at 
contract signing — such as budgets for the first year, assessments of building condition and performance 
indicators of tenant satisfaction — were not completed until well into the first year. As a result, the Department 
will not be able to carry out a complete assessment of the contractor’s performance in the first year and thereby 
get full assurance that it is receiving the quality and quantity of service to which it is entitled. 


Background and other observations 


18.3 Public Works and Government Services Canada, through its Real Property Services Branch (RPS), is the 
primary landlord of a significant proportion of the government’s real property holdings. Its inventory amounts to 
about 6 million square metres of space — from office buildings, laboratories and warehouses to the Parliament 
buildings and other “national treasures”. It also administers 3,000 leases representing annual rent of $550 million. 
This makes the Branch one of the largest property managers in the country. 


18.4 In 1996, the Department embarked on an initiative called “Alternative Forms of Delivery” (AFD). A key 
element of the initiative was to contract out the management and operations of a portfolio of buildings, including 
services ranging from cleaning and the operation of building systems to maintenance, landscaping and repair 
projects. This initiative was a response to the government’s 1994 Program Review and served as a strategy for 
restructuring the Branch around its core business activities. The Branch also wanted to support the Canadian 
property management industry and reduce costs through this initiative. 


18.5 The Department contracted out to the private sector the operations and maintenance of approximately 
300 buildings with about 2.3 million square metres of office space. It expects to achieve about $20 million 
annually in cost savings by turning over the management of those buildings to the private sector. 


18.6 The buildings were regrouped by region and offered to the private sector in 13 separate contracts, with a 
total value of around $170 million. The same contractor won each of the 13 contracts. It offered jobs to almost all 
affected employees at full salary for a minimum of three years. About 465 employees accepted the transfer to the 
contractor. 


18.7 It is too early to assess whether savings the Department expects from this initiative will be achieved. We 
cannot measure at this time whether the Department will receive services at least equal to those it provided itself, 
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and receive them at lower cost. However, mechanisms were incorporated in the contracts that are conducive to 
achieving savings. We will follow up on these issues in future audit work. 


18.8 We encourage Public Works and Government Services Canada to continue to use “fairness monitors” for 
large and complex contracts. In a project of this scope and complexity, we did not expect to find that all 
contractual requirements would be fulfilled by the handover date. We did expect that the Department would have 
established an action plan to deal with the uncompleted deliverables. We have recommended that it develop an 
action plan with the contractor to deal with outstanding contractual issues, including target completion dates and 
respective responsibilities for resolving them. 


The Department has agreed with our recommendation. 
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(ES A LS 


Industry Portfolio 


Investing in Innovation 


Chapter 19 — Main Points 


19.1 This audit sought to determine if four grant and contribution programs, through which over $1.3 billion 
was spent over the last three years, were well designed to help improve Canada’s innovation performance. We 
audited the following programs: 


e Industrial Research Assistance Program — National Research Council (NRC) 

e Research Partnerships Program — Natural Sciences and Engineering Research Council (NSERC) 
e Networks of Centres of Excellence — NSERC 

e Technology Partnerships Canada — Industry Canada 


19.2 We expected that these programs would be based on a sound understanding of innovation performance 
problems in the economy. We found that while there is a wealth of information on various aspects of innovation 
performance in the economy, it is unclear what this information means when considered together. Moreover, we 
found that management has not defined what specific innovation performance problems these programs are 
supposed to address, nor what specific results are expected from them toward promoting innovation. 


19.3 We also sought to determine if these programs were well managed and if management knew whether 
value for money was being achieved. We could not assure ourselves that many of the contributions under the 
Industrial Research Assistance Program and many of the grants under the Research Partnerships Program were 
properly supported. While the technological merit of the projects we examined had been well documented, there 
was often little explanation of the commercial or pre-commercial benefits expected from the projects, and of the 
need for government support. There are also important performance issues for which management has little 
information — in particular, on the commercial or pre-commercial results of funded projects. 


19.4 We concluded that due diligence had been exercised in the grants we audited under the Networks of 
Centres of Excellence program. We also concluded that the management of Technology Partnerships Canada 
(TPC) had exercised due diligence in making the contributions that we audited, with specific exceptions. TPC 
could make improvements in monitoring the progress and results of funded projects, and in reporting to 
Parliament on how it shares risks and returns with funding recipients. 


Background and other observations 


195 The government has made building a more innovative economy one of its policy goals. A number of 
recent government reports have referred to an “innovation gap”, meaning that Canada is not innovative enough 
compared with its main trading partners. These reports argue that weaker innovation performance lies at the heart 


of broader performance problems in the economy — particularly lower productivity in relation to the United 
States. 


19.6 However, our review of the issues suggests that the causes and effects of this gap are not straightforward. 
While innovation is undoubtedly an important factor in economic growth, assessing the actual innovation 

performance of the economy is a multifaceted challenge. Although a comprehensive assessment is still difficult to 
make, there is growing evidence that Canada’s performance lags behind that of its major competitors in a number 
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of specific ways. It is reasonably clear that better innovation performance depends on more than just increased 
spending on research and development; it involves supporting research and development with the activities needed 
to embed new technologies in the economy. It seems equally clear that spending on research and development is 
not the only determinant of the Canadian economy’s rate of productivity growth, and may not be the most 
important one. 


19.7 Promoting innovation in the economy is one of the principal objectives of the Industry Portfolio, which is 
made up of the organizations for which the Minister of Industry is responsible. The programs we audited focus on 
supporting research and development and account for the bulk of the grants and contributions made by the 
Portfolio toward that objective: 


e =The National Research Council delivers the Industrial Research Assistance Program, which helps 
small- and medium-sized enterprises develop and exploit technologies ($120 million in 1998-99), 


e The Natural Sciences and Engineering Research Council delivers the Research Partnerships Program 
with the objective of fostering interactions and partnerships between university researchers and other 
sectors in order to generate new knowledge and develop new expertise, and to transfer this new 
knowledge and expertise to Canadian-based organizations ($95 million in 1998-99). NSERC also 
delivers the Networks of Centres of Excellence program to improve Canada’s performance in science 
and technology, and to facilitate transfer of knowledge to those who can use it to advance Canada’s 
social and economic development ($47 million in 1998-99), 


e Technology Partnerships Canada is a special operating agency within Industry Canada. It is intended 
to promote the development and commercialization of innovative technologies that contribute to 
increasing economic growth and creating jobs and wealth ($250 million in 1998-99). 


The responses of Industry Canada, the National Research Council and the Natural Sciences and Engineering 
Research Council to our recommendations are included in the chapter. The two councils have agreed to act 
on all of our recommendations. Industry Canada has agreed to act on all but one of our recommendations. 
The Department has indicated that no additional measures are required to address our recommendation 
concerning the justification for the amount of its contributions. 
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SSS EF EE PE 


Fisheries and Oceans 


Pacific Salmon: Sustainability 
of the Fisheries 


Chapter 20 - Main Points 


20.1 The Pacific salmon fisheries are in trouble. Catches have declined overall, but the commercial catch has 
plummeted. The long-term sustainability of the fisheries is at risk because factors like overfishing, habitat loss, 
and declining ocean productivity have eroded the resource base. The result is a fisheries management crisis that 
has cast a cloud of uncertainty over the future of the salmon fisheries. Salmon fishing will continue, but more 
stringent controls are needed in the short term to ensure that salmon survive for the benefit of future generations. 


20.2 The management challenge for the Department of Fisheries and Oceans is to conserve existing stocks and 
rebuild those that are at low levels, while maintaining the viability of the fisheries. It will have to adapt its 
management regime to the new realities and gain the acceptance and support of stakeholders if it is to be 
successful. These will be difficult tasks to accomplish. 


20.3 The Department now needs to: 
e identify information needs and priorities; 
e review the consultation process and identify areas for improvement; 
e implement integrated fisheries management plans; 
e complete the downsizing of the fleet as a priority; 
e establish an independent allocation board; and 


e develop a better working relationship with the Province of British Columbia where fishery issues are 
concerned. 


Background and other observations 


20.4 The Department has already taken the first steps to address the challenges it faces. It has affirmed 
conservation as its primary objective to protect existing salmon stocks and rebuild the resource base. In 1998 it 
announced a new policy, A New Direction for Canada’s Pacific Salmon Fisheries, to direct its management 
activities in the future. The regional office has developed and is implementing its strategic plan for 1998-2001 
based on this policy. 


20.5 In addition to the management of larger stocks targeted primarily by the commercial fishery, the regional 
office has begun to focus more on the conservation of smaller stocks that are important to the Aboriginal and 
recreational fisheries. It has taken a conservation-based approach to fisheries management and is testing selective 
fishing methods and gear, in addition to implementing special programs such as fleet reduction and restructuring. 
However, there are areas that need attention if policy objectives are to be attained. The most important are 
improving regional databases and information management to enable it to manage Pacific salmon on a 
stock-by-stock basis, and resolving consultation problems to improve stakeholder relations and move toward 
forming partnerships to share management responsibilities and offset costs. 


20.6 The Department will need time to adjust to the new management regime. During this time, fisheries will 
be subject to closure to allow stocks to recover. Improvements cannot be expected to occur quickly, given that 
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salmon life cycles can extend to five years or longer and salmon need time to adjust to changes in their freshwater 
and marine habitats. Finally, the future sustainability of the fisheries will depend not only on the Department’s 
ability to implement its New Direction policy but also on its success in enlisting the help of stakeholders and the 


provincial government to share the workload. 


The responses of Fisheries and Oceans to our recommendations are included in this chapter. The 
Department agrees with all 13 recommendations and notes that initiatives are under way in several areas. 
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SE ESE AI 


Financial Information Strategy 


Departmental Readiness 


Chapter 21 - Main Points 


74) Fa We found that most departments are just starting to focus on the Financial Information Strategy (FIS) 
despite the fact that the target implementation date for FIS, 1 April 2001, is less than two years away. At the time 
of our audit, while most departments had implemented new financial systems, they still had a lot of work to do to 
prepare these financial systems and to implement the accrual accounting policies to meet FIS requirements. 
Strategies for preparing managers to use the information provided by FIS to strengthen management decision 
making are not yet in place. We are also concerned that FIS is not seen as a high priority by senior managers. 


21.2 in our view, the Treasury Board Secretariat, overall project manager for FIS, will need to assume a 
greater leadership role, drawing on some of the lessons learned in dealing with the Year 2000 problem. In 
particular, the central FIS Project Office will need to put in place and keep current an updated overall 
implementation plan, and use appropriate risk management capabilities, monitor implementation by departments 
and intervene constructively if problems arise. In addition, it will need to provide departments with required 
accounting policies and manuals, which are currently being developed, and assist departments in developing an 
appreciation of the use of FIS in day-to-day management. 


21.3 Public Works and Government Services Canada (PWGSC) effectively carried out the development and 
implementation of new central systems. PWGSC and its major partners, the Secretariat and Revenue Canada, 
worked together to conclude a successful pre-production pilot of those new systems. We found that the 
Department and its partners established a detailed plan and risk management capability, monitored progress and 
intervened when necessary, and met key target dates. 


Background and other observations 


21.4 As we reported last year in our first report on FIS (September 1998 Chapter 18), the government is 
making sweeping changes in the type of financial information provided to decision makers. Under the Financial 
Information Strategy, new financial systems throughout government are being implemented. But FIS is much 
more than the renewal of aging financial systems. The Strategy is also designed to help the government strengthen 
significantly its management of business lines and its accountability to Parliament. To do this, FIS will generate 
full accrual accounting information similar to that used by the private sector and integrate this information into 
day-to-day decision making of departmental managers. The Office of the Auditor General continues to support 
fully these stated objectives of FIS. 


21.5 The push for improvements in financial information to support government decision making started 

in 1962 with the Royal Commission on Government Organization (Glassco Commission). FIS was officially 
launched in 1989 and, after several false starts, the project was revitalized in 1995. A goal of April 2001 was 
published as the target date for implementation. We noted in our research that in some other jurisdictions, such as 
the Province of Alberta, the United Kingdom, Australia and New Zealand, financial management reform has 
progressed more rapidly. 


21.6 Parliament’s Standing Committee on Public Accounts held a hearing on Chapter 18 in October of 1998. 
The Committee called for the Treasury Board Secretariat to proceed with “all diligence and speed” in pursuing the 
option discussed in the chapter of changing the appropriations process to focus on resources consumed to achieve 
results rather than simply resources acquired. We saw this change in the appropriations process as a necessary step 
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in creating demand by departmental managers for full accrual information to strengthen day-to-day management 
and decision making. The Committee also called for the Secretariat to monitor the progress of FIS across all 
departments and agencies and to provide Parliament with better estimates of the total implementation costs. 
Estimates of FIS costs are still largely unknown or undetermined; only 11 of 24 departments that we surveyed had 
partial or full cost estimates. 


247 One of the key aspects of successfully delivering a major initiative like FIS is the cultural change 
required at the management level to fully participate in and reap the benefits from the new approaches to informed 
decision making that FIS brings. The importance of change management has been well publicized in industry and 
in other government jurisdictions. Looking ahead, a major challenge for the Secretariat will be to provide 
guidance and lessons learned to departments with regard to this important issue. 


The Treasury Board Secretariat’s responses to our recommendations are included in the chapter. In line 
with the recent decision to increase its focus on implementation of the Strategy, the Secretariat will monitor 
departmental progress and take into consideration our recommendations. 
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Attributes of Well-Managed 
Research Organizations 


Chapter 22 — Main Points 


22.1 We found that well-managed research organizations share a number of attributes that we have grouped 
under four themes: 


e They focus on people, recruiting, developing and retaining the right mix of talent in a positive and 
supportive environment. 


e They show leadership, aligning themselves with the needs of those who depend on them for results, 
achieving buy-in of the vision, values and goals, and undertaking the right research at the right time 
and at the right investment. 


e They manage research to ensure excellence and results, the leveraging of resources, and the capture 
of organizational learning. 


e They strive for a high level of organizational performance, being widely known and respected, and 
meeting the needs of those who depend on them for results. 


22. We developed the attributes following research and consultation with research managers in Canada and 
the United States. The attributes are not a recipe for action but rather statements of the direction that management 
action should take. They are supplemented by examples of practices used by organizations we visited. 


22.3 While the attributes describe ideal outcomes, and are therefore not attainable, they are stated so that 
progress toward the ideal is observable and measurable. In our view, the extent to which an attribute is 
demonstrated by an organization is a measure of the quality of its management. 


Background and other observations 


22.4 The federal government invested nearly $5.5 billion in science and technology (S&T) in 1998-99. Of 
that amount, $3.2 billion was spent by federal research organizations, with the balance being paid to outside 
organizations to undertake research and development and related scientific activities. 


Depbers In Science and Technology for the New Century: A Federal Strategy (1996), the federal government 
recognized the need for better management of S&T activities and provided new governance mechanisms and a set 
of general operating principles to improve S&T management across and within departments and agencies. Our 
work complements and reinforces the Strategy by describing what good management should look like in a 
research organization. 
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Involving Others in Governing 


Accountability at Risk 


Chapter 23 — Main Points 


23.1 We found a total of 77 new governance arrangements across the federal government, involving annual 
expenditures totalling over $5 billion. Federal investments in some arrangements are quite small, such as the 
Canadian Industry Program for Energy Conservation, but others involve federal commitments of billions of 
dollars, such as the Canada Infrastructure Works Program. 


23.2 Under these arrangements, the federal government involves external partners in the planning, design and 
achievement of federal objectives, replacing delivery by federal employees, contractors or agents. These partners 
are not accountable to ministers and Parliament. 


23.3 These initiatives, if properly implemented, have the potential to improve the delivery of federal programs 
and services. However, many of the new governance arrangements we examined have been put together in an ad 
hoc manner that puts accountability to Parliament at unnecessary risk. Parliament has limited means under these 
arrangements — in some cases no means — of holding the government to account for the federal functions 
performed or the federal objectives to be achieved. Good will and trust alone, while essential in all arrangements, 
are not adequate insurance for continued success in the long term. 


23.4 For these new arrangements, the government does not have in place a consistent and generally accepted 
governing framework that safeguards the essential principles of our parliamentary system. Nor has it been 
adequately capturing and communicating the lessons being learned in these new approaches. In our view, the 
federal government remains accountable to Parliament for the use of federal tax dollars, assets and authorities, no 
matter what tools it uses or arrangements it puts in place with partners to achieve its public objectives. 


23.5 Parliament and the public need to be consulted on the development of an adequate governing framework 
that will reconcile new governance arrangements with accountability to Parliament for the exercise of federal 
functions by parties outside the federal government. 


Background and other observations 


23.6 Over the last decade, the government has significantly increased its use of external partners in innovative 
arrangements to deliver federal programs and services to Canadians. In some cases, these arrangements have 
diffused federal power, by drawing outside parties into the process of actually governing Canadians in important 
areas of public policy that were once the sole domain of the federal government. 


23.7 The new governance arrangements we examined use a wide variety of approaches to program and service 
delivery. Provision for ensuring good governance and accountability to Parliament and the public is very patchy: 
we found limited reporting of performance, many weak accountability mechanisms, and inadequate attention to 
transparency and protection of the public interest. These need to be fixed. 


23.8 The government needs to ensure that departments and agencies setting up new arrangements address the 
essential issues of credible reporting to Parliament and the public, effective accountability mechanisms, adequate 
transparency and protection of the public interest. The Treasury Board Secretariat’s leadership and commitment 
are needed in developing a governing framework and overseeing its use, recognizing that what constitutes 
appropriate and adequate specific provisions to address these issues will vary from case to case. 
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7a There is a balance to be struck between the independence these arrangements need to operate efficiently 
to achieve results and the need for adequate accountability. In our view, appropriate accountability to Parliament 
and the public is not incompatible with independence from government intervention in operational matters. 


The response of the Treasury Board Secretariat is included at the end of the chapter. The Secretariat 
endorses the elements of the governing framework we propose, stresses the need for flexibility in their 
application and acknowledges the need for improvement in some areas. The Secretariat mentions several 
steps it is taking to address issues identified in this chapter. 
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The Canadian Adaptation and 
Rural Development Fund 


An Example of Involving 
Others in Governing 


Chapter 24 - Main Points 


24.1 The Canadian Adaptation and Rural Development (CARD) fund is a new governance arrangement 
developed by Agriculture and Agri-Food Canada to help farmers, producers and processors adapt to the new 
business realities of farming. Under the arrangement, projects to support agricultural adaptation are selected 
largely by the agriculture industry for funding. 


24.2 We found that a reasonable balance had been struck between giving industry councils the freedom to 
make the best decisions and respecting the public purpose of the funds. In designing this arrangement, the 


Department developed a number of good practices. However, we also found some areas that need to be improved. 


24.3 In conjunction with the Department, we developed an instrument to assess the capabilities of the 
adaptation councils. Pilot results suggest that the CARD councils rate well in most of the required competencies. 
We determined that the instrument could be useful for assessing the capabilities of parties in this new governance 
arrangement and perhaps in others. 


The Department accepts our findings and has committed to act on our recommendations, although no 
details have been provided. 


Report of the Auditor General of Canada — November 1999 


NS La LE aT A EA 


Preparedness for Year 2000 


Final Preparation 


Chapter 25 — Main Points 


PE | The government has made significant progress in preparing its systems that support government-wide 
mission-critical functions for the Year 2000 computer problem. The Treasury Board Secretariat reported that work 
on government-wide mission-critical systems was 99 percent complete at July 1999. Our audit supports the overall 
99 percent completion rate for those systems. According to its plans, all government-wide mission-critical systems 
were to be ready for Year 2000 by 31 October 1999, two months before the new millennium. 


25.2 Health Canada and the Atomic Energy Control Board have established Year 2000 requirements for 
licensees of medical devices, nuclear power reactors and radioactive devices. Some follow-up is needed for 
medical and radioactive devices but the licensees for active nuclear power plants have met the requirements. 


25.3 Measures are being put in place for contingencies and national emergencies. Although contingency 
procedures have largely been defined, departmental contingency planning needs more work. 


25.4 We concluded that the government needs to remain vigilant to keep any Year 2000 disruptions to a 
minimum. 


Background and other observations 


25.5 Year 2000, the two-digit year code problem, has been a cause of concern to industry and governments 
around the world. The estimated costs of addressing this problem run as high as US$800 billion worldwide. In 
August 1999, the federal government estimated the costs of its Year 2000 projects at $2.2 billion. According to the 
Treasury Board Secretariat, the final costs could reach $2.5 billion. 


25.6 In 1997 we audited the government’s preparedness for Year 2000, and again in 1998. Our 1997 Report 
noted our concern about the slow pace of Year 2000 work; in 1998, we remained very concerned that some 
essential services might be interrupted in 2000. Most of our recommendations have been accepted and 
implemented by the government. 


25.7 In 1999, as we completed our work in individual departments and agencies, we reported our findings to 
management and suggested actions to consider. That additional step was taken to provide more time for 
departments and agencies to act. 


25.8 We verified the government’s Year 2000 progress information as reported by the Secretariat against the 
information in its supporting files and we further reviewed departmental documents for seven government-wide 
mission-critical functions in six organizations. Our verification showed no substantive differences from the 
information reported by the Secretariat. 


25.9 In the departmental contingency plans we reviewed, we found that some key components were not 
complete or lacked specific details. In particular, plans for testing were weak and few organizations planned to 
complement the National Contingency Planning Group validation exercise with other tests of their contingency 
procedures. 

25.10 We have identified several issues that will require action beyond 1999, and have recommended measures 
for the government to take or to consider. They include moving to comply with government date standards; 


maintaining and updating valuable information bases developed from Year 2000 projects; and looking out for Year 
2000 pitfalls after January 2000. 


The government’s responses to our recommendations are included in the chapter. The government agrees 
with our recommendations and the responses identify the action that it will take to address them. 
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National Defence 


The Proper Conduct of 
Public Business 


Chapter 26 — Main Points 


26.1 National Defence has taken positive steps in each of the areas we reviewed in this audit: active support of 
ethical conduct, internal control and audit, and follow-up on complaints. However, each of these areas requires 
further improvement. 


26.2 The Defence Ethics Program, initiated by the Department in 1994 as a key response to ethical problems 
that had surfaced in recent years, has not yet been fully implemented throughout the Department and the Canadian 
Forces. The central design of the program has been completed, ethics components have been added to some 
training courses and ethics training for new recruits is progressing at a satisfactory pace, but other key elements 
are still not in place. In the military services there is a lack of commitment to the program, and its integration into 
personnel systems varies widely. The Department needs to move forward and make the program fully operational 
as soon as possible. To do so, it needs to develop an action plan with clear dates and deliverables. 


26.3 We also found weaknesses in the departmental control systems, including internal controls, internal audit 
and the military police. There is evidence that in some areas, controls over financial and materiel resources have 
weakened. While the central internal audit group has recently completed several studies of risks associated with 
delegated resource management, we are concerned that internal audit resources have fallen below prudent levels 
and that decisions about the level of compliance audit work necessary to detect and deter fraud and abuse are not 
based on adequate risk analysis. A few local commanders do not fully understand the new role of the Canadian 
Forces Provost Marshal in setting policy and standards for the military police. This has resulted in instances of 
interference. 


26.4 We examined how the Department responded to complaints directed to senior departmental management 
in National Defence headquarters. Allegations of such abuses of resources as unauthorized upgrading of official 
residences and misappropriation of government property have not always been dealt with adequately. Corrective 
action has been inconsistent and remedial action is slow. We have recommended that the Department track all 
complaint cases and ensure that they are resolved expeditiously. 


Background and other observations 


26.5 In recent years, National Defence has suffered from several highly publicized breakdowns in 
management and leadership conduct, most notably the subject of the Somalia Inquiry, and others related to misuse 
of resources. Over the past five years, the Department has also undergone extensive change: its budget has been 
reduced by almost 25 percent, an entire level of management has disappeared, and management has been 
substantially decentralized. This has posed significant challenges to the maintenance of management controls and 
the reform of the Department. 


26.6 In response to identified problems, senior officials in the Department directed the development of the 
Defence Ethics Program in February 1994. The program that was developed was approved in December 1997. It is 
based on the Statement of Defence Ethics, which consists of principles (respect, service and obedience to lawful 
authority) and obligations (integrity, loyalty, honesty, courage, responsibility and fairness). The program is 
intended to be implemented from the top down by group principals at headquarters and by the navy, army and air 
force chiefs of staff. 
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26.7 The Department’s strategy for implementing the Defence Ethics Program includes raising awareness, 
providing ethics training for military members and civilian employees, encouraging discussion and open dialogue 
and finding means to deal with ethical issues. It also includes assessing areas of risk to identify specific ethical 
problems. One of our objectives in this audit was to determine the extent to which the Defence Ethics Program has 
addressed the problems identified so far. 


26.8 In addition to the Defence Ethics Program, the. Department continues to rely on controls such as internal 
audit, military police and management checks on compliance. We examined these areas as well. We also assessed 
whether National Defence responds appropriately to complaints about misuse of resources or misconduct. 


The Department responded positively to our recommendations and agreed to make several improvements. It 
said it would re-emphasize the responsibilities of managers for implementing the Defence Ethics Program 
and would assist them in making plans. The Department agreed to strengthen the risk analysis required to 
guide audits in the individual military services. It hopes to use automated data analysis to analyze 
transactions in order to identify irregularities. Finally, the Department said it is now tracking responses to 
complaints of abuse and is confident that most assessments will be complete within one year. 
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National Defence 


Alternative Service Delivery 


Chapter 27 - Main Points 


27.1 The Alternative Service Delivery (ASD) program at National Defence — a search for new and better 
ways of providing government services — is still in its early stages and has made gains at a slower rate than the 
Department had initially expected. Unconfirmed departmental estimates suggest that savings are currently about 
$68 million. Given the Department’s $3.5 billion annual expenditures on support services, considerable scope for 
savings remains. In 1996 it projected that annual savings would reach $200 million by 1999. Based on results to 
date, however, the Department has revised this projection to $175 million a year by 2004. 


27.2 Many of the business case analyses for the 14 projects we audited were poorly done. Options were not 
always adequately assessed or the best option chosen. Personnel appeared to lack the necessary skills to undertake 
analyses. The Department has taken steps to improve its management of more recent ASD projects and believes it 
has corrected the earlier shortcomings. 


Za The government lacks an adequate policy framework for “partnering” with the private sector and 
contracting out large service programs. In particular, the $2.8 billion NATO Flying Training in Canada contract: 


e was let without competition, contrary to government contracting policy and regulations, thus 
forgoing the benefits of price competition; and 


e = did not follow Public Works and Government Services Canada’s profit policy and guidelines for 
sole-source contracts, and profit markups were not supported by adequate analyses. 


27.4 Inflexible contract arrangements resulted in payments for unused training capacity. For example: 


e The Meaford Area Training Centre is operated under a $40 million five-year fixed-price contract. 
However, it was used at only 43 percent capacity in 1998. 


e The Canadian Aviation Training Centre, Portage-la-Prairie operates a flying training program under a 
$165 million contract; it was substantially underutilized during the first six years. 


Background and other observations 


27.5 According to National Defence, the aim of its ASD program is to provide a framework for departmental 
managers to pursue best value for the defence dollar in non-core activities. Through the program, it hopes to 
identify and use the most cost-effective ways of delivering support services, which constitute about one third of 
the Department’s $10.3 billion budget. We noted that many of the activities being considered for the ASD 
program have already achieved savings through downsizing and re-engineering. 


27.6 When the Department launched the ASD program in 1995, it set a goal of saving $200 million a year by 
1999 and $350 million a year by 2001. We audited 14 of its 40 active ASD projects. We audited only government 
actions and our observations imply no criticism of any third party supplying services to the government. 


The responses of National Defence, the Treasury Board Secretariat and Public Works and Government 
Services Canada are included in this chapter. National Defence is taking action to address all our 
recommendations. In particular, the Department is devoting a higher level of management attention to ASD 
projects and ensuring that staff are adequately trained. 
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With respect to our concern about the lack of an adequate policy framework for “partnering” with the 
private sector, the Treasury Board Secretariat and Public Works and Government Services Canada are 
currently participating in an interdepartmental initiative to reform procurement that will include, among 
other matters, work on guidance for large, multi-year service contracts. 


National Defence believes that the NATO Flying Training in Canada program is innovative and will provide 
good-quality pilot training at a lower cost than the current training system or any other training option in 
the foreseeable future. 
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Canadian International 
Development Agency 


Financial Controls Over Projects 


Chapter 28 - Main Points 


28.1 CIDA’s financial controls are effective in monitoring and controlling the flow of funds for projects, and in 
tracking the financial status of projects — that is, what has been spent and what remains to be spent. However, 
reports on the progress of projects often do not compare what has been accomplished with what was planned. This 
makes it difficult to determine the rate of progress or to analyze the financial implications of changes from 
expected progress. The result is a risk that project scope may have to be reduced or the budget increased to 
accomplish the intended objectives. 


28.2 CIDA uses audits as a means of ensuring that its executing agents are respecting the financial conditions 
in its contracts with them. These audits have generally indicated that expenditures by executing agents are largely 
in accordance with contract conditions. In the contracts we audited, the amounts identified as possible adjustments 
because of non-compliance with contract conditions were relatively small — $32.2 million out of about 

$932 million. However, CIDA’s audits have also identified some persistent problems of non-compliance with 
contract conditions. CIDA has released (that is, waived) some dollar adjustments identified as amounts to be 
repaid by the executing agents as a result of non-compliance. The rationale for doing so was documented in most 
cases but, in our view, the reasons given did not always justify the releases. From our sample, we estimated these 
amounts to be $12 million to $13 million over the four years of reports we examined. 


28.3 We believe that the Performance Review Branch needs to be more active in auditing whether the system 
of financial controls is functioning as intended. 


Background and other observations 


28.4 CIDA is responsible for managing about $1.8 billion of Canada’s international assistance. Just over 
$1 billion represents contributions to third parties, usually referred to as Canadian executing agents (CEAs), to 
deliver development assistance projects. Executing agents may be Canadian or developing country institutions, 
provincial governments and their organizations and agencies, or Canadian private sector firms. 


28.5 CIDA has put in place a number of financial controls to support the management of its aid projects. At 
the centre of project management is the contract or the contribution agreement with the CEA. CIDA uses audits to 
ensure that CEAs are complying with its contracts and contribution agreements. In this audit, we wanted to see if 
CIDA’s financial controls for projects were functioning as intended. We also wanted to examine the actions taken 
by CIDA to deal with the results of its compliance audits of these projects. 


28.6 Although we noted problems of non-compliance with contracts, in nearly all the contracts we examined 
we also observed that once there was agreement between CIDA and a CEA that money should be repaid, CIDA 
recovered the amounts owing. Information generated by the audits commissioned by CIDA was being assembled 
and analyzed, but was not presented to senior program management for action. 


The Agency has accepted our recommendations and agreed to take action to address the issues raised. 
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EL I 


Federal Support of 
Health Care Delivery 


Chapter 29 - Main Points 


29.1 The federal government is not in a position to determine what its total contribution to health care really 
is. Federal funds are transferred under the Canada Health and Social Transfer in a block, providing provinces and 
territories with the flexibility to allocate these funds among health care, post-secondary education, and social 
assistance and social services. As a result, Parliament and the general public do not have a clear picture of the 
amount of federal funding directed to health care. 


DOD Health Canada strives to administer the Canada Health Act in a non-intrusive manner. This approach has 
not brought about the speedy resolution of non-compliance issues and differences in interpretation of the Act. 

A new approach, using the provisions of the Social Union Framework Agreement, offers a process for avoiding 
and resolving disputes. 


29.3 Deficiencies in the Department’s annual reports to Parliament compromise their usefulness: Parliament 
cannot readily determine the extent to which each province and territory has satisfied the criteria and conditions of 
the Canada Health Act. When the Department cannot provide this information in its reports, it should clearly 
explain the reasons. 


Background and other observations 


29.4 The delivery of health care is a primarily provincial/territorial responsibility. However, the federal 
government administers a significant piece of legislation in this area, the Canada Health Act. To many Canadians, 
the Canada Health Act provides for a health care system that helps to define this country. It articulates health care 
as a basic right and describes the features of the health care system. 


29.5 The Canada Health Act establishes five criteria and two conditions as well as extra-billing and user 
charge provisions. All of these must be met if a province or territory is to receive the full federal cash contribution 
under the Canada Health and Social Transfer. The five criteria mean that regardless of where people live in 
Canada, they have universal access to a comprehensive, publicly administered health care insurance plan that will 
cover them if they move to another province and when they travel in Canada. With certain limitations, it also 
covers them when they travel outside Canada. The two conditions of the Act require provinces and territories to 
supply information that the federal government may reasonably require and to publicly recognize federal transfers. 


29.6 Federal funding is provided to provinces and territories and other organizations to assist them in carrying 
out their health care mandates and related health activities. Health Canada is responsible for the administration of 
the Canada Health Act and other programs in health and related areas, including recent initiatives to renew health 
care and strengthen health information and technology. 


DO] We found that Health Canada does not have the information it needs to effectively monitor and report on 
the extent of compliance with the Canada Health Act. The only departmental evaluation undertaken in this area 
was limited in scope, and it was five years before results were reported to Parliament. 


29.8 The federal government is funding efforts to meet a wide range of information needs. Information is a 
critical tool for allocating resources and reporting on how well the health care system has served the public. This 
work presents major challenges that all parties concerned with national health information need to manage 
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carefully. In addition, Health Canada needs to move expeditiously to meet its commitment to report on the 
performance and effectiveness of its programs. 


29.9 The federal government has established the Health Transition Fund to support provinces in undertaking 
pilot projects to assess ways in which Canada’s health care system can be improved. It has also introduced the 
Health Infostructure Support Program to help organizations involved in health care services further test and assess 
their use of information technologies. We found weaknesses in the management of both initiatives that, in our 
view, could compromise their usefulness as tools for helping planners to make sound decisions on health care 
delivery. 


Responses to our recommendations from Health Canada and the federal government are included in the 
chapter. Health Canada has agreed to take corrective action on those recommendations directed to the 
Department. The federal government is committed to improving information generally but has not 
responded to our recommendations on the provision of specific information. 
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Sole-Source Contracting 
for Professional Services 


Using Advance Contract 
Award Notices 


Chapter 30 - Main Points 


30.1 This chapter examines the government’s practice of awarding sole-source contracts for professional 
services. It also examines a mechanism known as an Advance Contract Award Notice (ACAN), used widely by 
departments to advertise their intention to award sole-source contracts to a specific supplier. 


30.2 The principles of accessibility, competition, fairness to suppliers, transparency and best value lie at the 
core of government contracting policy. The contracting regulations require that all contracts be let through a 
competitive process, with certain very narrowly defined exceptions. When the contract is needed in an emergency, 
when the value is small, when it is not in the public interest to solicit bids (for example, if national security is 
involved) or when there is only one supplier who can do the work, the contract can be let without competition on 
what is called a sole-source basis. Almost 90 percent of the 50 sole-source contracts we examined did not fall 
under any of the exceptions or did not have adequate evidence of doing so and hence ought to have been 
competitively tendered. As in last year’s audit of sole-source contracts for professional services, we concluded 
that the process of awarding most of the contracts audited in this year’s sample would not pass the test of public 
scrutiny. 

30.3 We concluded that the ACANs associated with these sole-source contracts added transparency to directed 
contracting, because ACANs are publicly advertised (compared with the nearly 40 percent of sole-source contracts 
that are let without any public notice) and can be challenged before their expiry date. However, in our view the 
challenge process is flawed and discourages potential suppliers from submitting challenges. Only 35 of the 522 
contracts that we reviewed had been challenged and only 4 of the challenges were accepted. We found the 
following: 


e The information the ACANSs provide is often vague and they are challenged infrequently. 
Furthermore, many ACANSs are posted for less than the recommended 15 days. There is no policy 
defining the challenge process, nor any criteria for judging the validity of a challenge. Those who 
decide whether a challenge is valid are, for the most part, the same departmental officials who 
originally decided that the contract should not be open to competition. There is no recourse to appeal 
their decisions unless the contract 1s subject to trade agreements, in which case they may be appealed 
to the Canadian International Trade Tribunal. Mounting a challenge to an ACAN under these 
unfavourable circumstances requires the supplier to invest both money and good will, an investment 
that most find unattractive. 


e Based on the 50 sole-source contracts selected for detailed examination, in most instances the 
decision to contract is not well considered, the requirements are often defined only vaguely, pricing 
is not done with due regard to economy and often deliverables are not assessed against the original 
requirements of the contract. 


e The existing framework of contracting rules, policies and regulations for contracting is basically 
sound. However, the evidence shows that departments either do not understand this framework or in 
some instances choose not to follow it. 


Background and other observations 


30.4 In 1997, contracts of $25,000 and above for all types of services (including professional services) had a 
total value of $3.9 billion. Of that amount, $1.34 billion was for sole-source contracts. An ACAN was posted for 
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over half of those ($830 million) and the balance ($510 million) were entered into without public notice of the 
government’s intent to issue a sole-source contract. 


30.5 Our December 1998 Report (Chapter 26 — Contracting for Professional Services: Selected Sole-Source 
Contracts) noted that the process of awarding most of the contracts in the sample we audited that year would not 
stand the test of public scrutiny. In May 1999, the Standing Committee on Public Accounts supported the audit 
findings and expressed concerns about departmental practices as they relate to ACANSs. 


30.6 The objective of the current audit was to assess the use of sole-source contracts and ACANs in 
professional services contracts by National Defence, Human Resources Development Canada, the Canadian 
International Development Agency and Industry Canada, and by Public Works and Government Services Canada 
on their behalf. Our audit included all 522 ACANs issued by, or on behalf of, the four departments in 1998, and 
examined in detail a sample of 50 of these. 


30.7 We have made recommendations designed to encourage and strengthen the accountabilities in 
departments and agencies for the exercise of delegated contracting authority by managers. Additionally, to 
strengthen the challenge process for contracts, including contracts let using ACANs, we have recommended that 
the government clarify the due process rights of contractors and establish an independent appeal mechanism for 
contractors. 


The Treasury Board Secretariat has indicated that the Treasury Board’s contracting policies are based on 
the strong values and principles of competition, openness, equal access, transparency, fairness and best value 
for Canadians. It explains that the Board’s role is to establish these policies and that departments, in turn, 
are accountable to their ministers and to parliamentarians for implementing them. The Secretariat commits 
to introducing a program of training and certification for procurement specialists as well as implementing a 
monitoring framework for evaluating contracting activities. While Treasury Board policy encourages 
contract review mechanisms and internal audits, the Secretariat does not believe that a mandatory policy 
requirement for a contract review mechanism in departments is necessary, nor does it support the 
recommended scope for internal audits of sole-source contracts. The Secretariat does not believe that an 
independent appeal mechanism for contractors that deals with contracts lying outside the purview of the 
Canadian International Trade Tribunal is necessary. 
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Department of Foreign Affairs 
and International Trade 


Delivery of Capital Projects 
in Four Missions 


Chapter 31 - Main Points 


31.1 This audit examined how the Department of Foreign Affairs and International Trade planned and 
implemented six capital projects at four missions in Seoul, New Delhi, Geneva and Bangkok. Capital costs for 
these missions totalled about $75 million, representing approximately 20 percent of the capital expenditures 
planned by the Department over the five-year life of its Long-Term Capital Plan. 


31.2 Delivering capital projects outside of Canada entails significant additional risks, difficulties and 
challenges not experienced in the delivery of capital projects domestically. 


31.3 The audit confirmed that valid reasons existed for initiating each of the projects and that users are 
generally satisfied with their new accommodation. Overall, the projects were delivered within budgets and project 
schedules. In general, contracts were awarded on a competitive basis and change orders were well managed. We 
noted several positive initiatives to address environmental concerns. 


31.4 However, we identified weaknesses in the preliminary planning of the projects. Indicative project 
estimates were incomplete and unreliable. The preliminary cost estimates of three projects increased by 

$38 million, representing increases ranging from 64 percent to 153 percent over their initial estimates. In one case, 
a poor cost estimate may have resulted in an uneconomical expenditure of $15 million. We estimate that, over the 
last five years, the opportunity cost of this payment totals $8 million. 


31.5 We are also concerned with the lack of rigour of the Department’s analysis of options in support of their 
recommendations to the Treasury Board. We found instances where the Department failed to document the results 
of its analysis of various options that may have realized savings totalling at least $7 million over a 20-year period. 
We also noted weaknesses in the methodology used by the Department to support its investment decisions, and the 
lack of documentation to support a particular option. 


31.6 The quality of reporting to the Treasury Board to explain project delays and budget increases needs to be 
improved. 


SL In summary, the magnitude and frequency of increases in preliminary cost estimates and other problems 
identified by the audit seem to indicate that systemic weaknesses exist in the Department’s planning of capital 
projects and these need to be addressed. 


Background and other observations 


31.8 The Department of Foreign Affairs and International Trade manages a significant and diverse portfolio of 
office and residential accommodation, encompassing property in 160 locations in over 100 countries. The 
estimated value of these Crown-owned properties abroad is $1.5 billion to $2 billion and annual leasing and 
capital expenditures total approximately $110 million and $60 million respectively. The Department and other 
government departments use the facilities to deliver their programs. 


S18 This audit reaffirmed our opinion that the government’s administrative policies for managing capital 
projects are sound, but problems persist in their application. 


The Department of Foreign Affairs and International Trade agrees with the chapter’s recommendations and 
has developed an action plan that addresses our concerns. 
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Follow-up of Recommendations 
in Previous Reports 


Chapter 32 - Main Points 


32.1 The majority of our reviews this year depict an improvement in the efforts made by management to take 
positive and effective remedial action. In other cases, we are concerned with the slow progress on fixing the 
problems — for a variety of reasons, as outlined in the various segments. 


32.2 The Department of Finance has gone to considerable effort in trying to address and resolve the issues we 
raised in our 1997 chapter about its equalization program. The changes enacted through Bill C-65 had a 
significant financial impact on the federal government and resulted in payments being more consistent with the 
principles of the program. 


52.3 At Human Resources Development Canada, the Department expended considerable effort to improve 
service to clients and manage the Canada Pension Plan Disability program more effectively. It took action in each 
area discussed in our 1996 chapter. 


32.4 The RCMP Public Complaints Commission has undertaken initiatives to address our previous audit 
concerns about reducing its backlog of cases and streamlining its complaint review procedures. However, it needs 
to further reduce its backlog, develop a communications strategy, restructure its information system and work with 
the RCMP to meet its review turnaround target. 


32.5 Our follow-up of Canada’s Export Promotion activities found that Industry Canada and Foreign Affairs 
and International Trade have made significant progress on many operational matters; however, progress is much 
less evident in areas that would improve accountability for results. As well, Foreign Affairs and International 
Trade advised the Standing Committee on Public Accounts that it would propose a realistic and feasible service 
charging scheme, but the departments have decided not to implement charging for services. While the departments 
believe that the measures they have taken are sufficient, we are concerned that the progress we have observed will 
not be sustained if our other recommendations are not implemented. 


32.6 In our follow-up of financial management and control at Foreign Affairs and International Trade, we 
noted a lack of urgency in implementing a number of our recommendations; however, where the Department had 
focussed its efforts, it had made considerable improvements. 


32.7 The Treasury Board Secretariat has taken certain steps to respond to our recommendations on Renewing 
Government Services Using Information Technology. The Secretariat has made good progress in its monitoring 
and measurement of government-wide information management and information technology initiatives. However, 
progress over the past three years has been slow and, to date, full implementation of action to address our 
recommendations has been limited. 


32.8 A number of the original chapters were reviewed by the Standing Committee on Public Accounts. We 
feel that this has been instrumental in bringing about necessary changes. 
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Other Audit Observations 


Chapter 33 — Main Points 


S851! The Auditor General Act requires the Auditor General to include in his Reports matters of significance 
that, in his opinion, should be brought to the attention of the House of Commons. 


3.2 The “Other Audit Observations” chapter fulfils a special role in the Reports. Other chapters normally 
describe the findings of the comprehensive audits we perform in particular departments, or they report on audits 
and studies of issues that relate to operations of the government as a whole. This chapter reports on specific 
matters that have come to our attention during our financial and compliance audits of the Public Accounts of 
Canada, Crown corporations and other entities, or during our value-for-money audits. 


335 The chapter normally contains observations concerning departmental expenditures and/or revenues. The 
issues addressed generally involve failure to comply with authorities, and the expenditure of money without due 
regard to economy. 


33.4 Observations reported this year cover the following: 
e secret commissions/kickbacks for refueling military vehicles; 
e the lack of financial control and management systems at CORCAN; and 
e the Employment Insurance Account surplus. 


33.5 Although the individual audit observations report matters of significance, they should not be used as a 
basis for drawing conclusions about matters we did not examine. 
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Chapter 11 


Agriculture Portfolio 


User Charges 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 
Accountants. The numbered paragraphs in bold face represent recommendations. 


Table of Contents 


Main Points 


Introduction 


Focus of the audit 
Observations 


Implementation of Government Policy Is Incomplete 
Improvements needed in program costing 
Service standards are not widely used 
Are the impacts of user charges well understood? 
Exploring alternatives before establishing user charges 
Consultation process has improved 
Developing alternative fee structures 
Changing fees is a lengthy process 
User charges are not well integrated into strategic planning 
Planning of user charges could be improved at the program level 
Guiding principles for user charges can help 
Determining private benefit and public good can be difficult 
User charges can provide more than just revenue 
Little review of performance 
Dispute resolution processes are not well known 
What opportunities are there to learn from experience? 
Reports on plans and on performance fail to inform adequately 


Moratorium on New Fees 
Impacts of the moratorium 
What action is possible? 


Conclusion and Recommendations 


About the Audit 


Inserts 


Status and Decisions Facing Each Organization 
Key Elements and Good Practices for Managing User Charges 


Case Studies 


The Slow Pace of Change Has a Price 

Integrating User Charges Into an Organization’s Strategic Planning 
Similar Programs — Dissimilar User Charges 

Cost Recovery: Benefits Achieved Beyond Revenue Generation 


Exhibits 

11.1 User Charge Revenues Vary in the Agriculture Portfolio 

11.2 Cumulative Effect of Federal User Charges Varies by Sector 

BS A Significant Advance in Understanding Cumulative Fee Impacts 
11.4 Information on User Charges Has Limited Usefulness 
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Agriculture Portfolio 


User Charges 


Main Points 


11.1 We noted a number of good practices in the way Agriculture and Agri-Food Canada, the Canadian Grain 
Commission and the Canadian Food Inspection Agency have designed and managed fees they charge to users of 
their services. But these organizations do not yet meet all the requirements of the government’s policy on user 
charges. Although in recent years they have made progress, they are losing momentum and user charges are not 
being updated as programs change. This could result in some user groups paying either more or less than their fair 
share. 


15 Our audit found that the organizations often view user fees primarily as a means of generating revenue, 
ignoring their potential to achieve other possible benefits. For example, the Canadian Food Inspection Agency 
often charges the same fee for inspections whether or not food inspectors have found deficiencies, even though an 
inspection that has found problems entails additional time. This provides no incentive for the industry to comply 
with regulations and thereby minimize the time government inspectors need to spend. 


ins The organizations we audited need to make major improvements in the way they report to the public, the 
industry and Parliament on their plans for user charges and on the results. Despite the fact that this is a contentious 
issue that has sparked a great deal of interest from Parliament, the organizations in most cases have not publicly 
explained why specific user charges were needed and how they would be applied. Nor have their reports provided 
enough information to allow readers to judge whether user charges are managed properly. 


11.4 The three organizations need to improve their management of user charges in a number of areas, 
including costing their services, assessing the impact of fees, establishing formal appeal processes and integrating 
user charges into their strategic planning. 


Background and other observations 


LIS In its 1997 Cost Recovery and Charging Policy, the federal government states that user charges should be 
implemented for “services that provide identifiable recipients with direct benefits beyond those enjoyed by the 
general public, unless overriding policy objectives would be compromised.” A key underlying principle is that 
user charges provide benefits much broader than the revenue generated, such as making services more responsive 
to users’ needs. 


11.6 Organizations in the agriculture portfolio charge for many different services. For example, the Canadian 
Grain Commission charges the grain industry for weighing and inspecting grain destined for export. The Canadian 
Food Inspection Agency charges importers and exporters to inspect plants and plant products for pests or diseases. 
Agriculture and Agri-Food Canada charges owners of cattle that graze on community pastures for services it 
provides. 


| I) User charges have been a focal point of much concern in the agriculture sector. Over the past five years, 
revenues from user charges in certain areas have increased by as much as 300 percent. Producers and the industry 
have objected to the way the fees were introduced and to the cumulative impact they have had. Some have 
contended that the fees were simply a tax. 


11.8 A moratorium on new fee increases was announced at the time the Canadian Food Inspection Agency 
was created, to be in effect until the year 2000. Officials of the three organizations have given many different 
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interpretations of not only how long the moratorium will last but whether it applies to other organizations in the 
portfolio. We note that the organizations we audited believe they are constrained from correcting even known 
inconsistencies among fees and responding to changes in their business environment. In our view, the period of the 
moratorium offers them an opportunity to reassess their current position, develop guiding principles and plan 
improvements. 


11.9 The organizations face specific challenges in that they recover fees from individuals they also regulate. It 
can be difficult to establish the amount that users of a service should be charged for the benefits they receive when 
the service also benefits the general public. There is a need for broader consultation, where possible, that takes 
into account the interests of both industry and the public. Those who pay fees want to influence how services are 
provided, and the organizations must ensure that the needs of all stakeholders are taken into account. 


11.10 The three organizations need to make key decisions on user charges and take action to resolve the 
following situations: 


e Recent changes in its operations and in the grain industry have made the existing fee structure of the 
Canadian Grain Commission no longer appropriate. 


e The Canadian Food Inspection Agency initiated fee increases but has not completed its plan for user 
charges. Consequently, commodity groups are paying different proportions of the Agency’s costs for 
similar service, which represents inconsistent treatment. 


e Agriculture and Agri-Food Canada has not explained adequately when, how and why user charges 
are implemented for each of its programs. Consequently, the reasons why some programs have user 
charges and other, similar programs do not remain unclear. 


The responses to our recommendations from Agriculture and Agri-Food Canada, the Canadian Food 
Inspection Agency and the Canadian Grain Commission are included at the end of this chapter. Each of the 
organizations has said that it will act upon the recommendations that apply to it. 
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- Introduction 


11.11 Federal user charges in the 
agriculture portfolio are not new. For more 
than 60 years, farmers have paid grazing 
fees to the Prairie Farm Rehabilitation 
Administration for their use of community 
pastures. And fees have been paid to the 
Canadian Grain Commission for almost 
90 years to inspect grain destined for 
export. 


11.12 +The major organizations in the 
agriculture portfolio who apply user 
charges are Agriculture and Agri-Food 
Canada (the Department), the Canadian 
Grain Commission and the Canadian Food 
Inspection Agency. User charges cover a 
wide range of services, from inspecting 
cattle slaughter operations to certifying 
product standards and quality, to 
administering loan guarantee registration. 
User charges for these organizations 
totalled $135.6 million in 1997-98, the 
seventh highest in the federal government, 
where the total was $3.937 billion. 


11.13. The proportion of their 
expenditures that the organizations 
finance through user charges varies 
considerably. The Canadian Grain 
Commission recovers close to its full 
costs. The Prairie Farm Rehabilitation 
Administration (a branch of Agriculture 
and Agri-Food Canada) recovers almost’ 
all direct costs of providing pastures for 
grazing and breeding services for cattle. 
At the same time, there are minimal or no 
user charges for many programs in the 
Department’s Market and Industry 
Services Branch and Research Branch. 


11.14 The government defines external 
user charges as “all non-tax revenues 
related to the provision of goods, 
regulatory and non-regulatory services, or 
rights and privileges, from all sources, 
whether generated in Canada or abroad, 
from persons or organizations external to 
the federal government.” 


11.15 In April 1997, the government 
revised its 1989 cost recovery charging 
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policy to take a less prescriptive approach. 
The policy outlines several broad 
principles that departments and agencies 
must apply to user charges. First, they 
should charge for “services that provide 
identifiable recipients with direct benefits 
beyond those enjoyed by the general 
public.” This is designed to ensure that the 
general taxpayer does not pay the full cost 
of services that provide direct benefits to 
specific individuals or organizations. 


11.16 The policy’s second principle 
emphasizes that the benefits of user 
charges can go beyond revenue generation 
to better and more client-oriented 
services. The policy also advances the 
notion that user charges can serve as a 
market test of demand for services and 
can help reduce excess demand for “free” 
services. 


11.17 Government-wide initiatives have 
had an impact on user charges in the 
agriculture portfolio. The Program Review 
exercise of 1994-95 required departments 
and agencies to review each of their 
programs, with the aim of eliminating 
programs that were not really needed, 
improving service to clients and 
re-engineering programs to reduce overall 
spending. The government suggested that 
expenditure reductions could be 
accomplished through program cuts or by 
finding other sources of revenue. Each of 
the three organizations we audited has 
made significant expenditure reductions 
since Program Review, in some areas by 
as much as 20 percent. 


11.18 Revenues from user charges have 
increased substantially for the Canadian 
Food Inspection Agency since 1994-95, 
while changing less significantly for the 
other two organizations in our audit (see 
Exhibit 11.1). These increases, as well as 
fee increases from other departments and 
levels of government, have been met with 
strong opposition from producers and 
industry. 

11.19 The key concerns of stakeholders 
are the way the fees were introduced and 
the cumulative impact of the fees. Many 


Federal user charges in 
the agriculture 
portfolio are not new. 
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Our audit asked the 
question, “How well 
are the organizations 
in the agriculture 
portfolio managing 
their external user 
charges?” 


Exhibit 11.1 


User Charge Revenues Vary in 
the Agriculture Portfolio 


[| 1994-1995 
1997-1998 


Source: Public Accounts of Canada 
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contend that user fees are simply another 
form of taxation. A few have said that 
user charges may distort the impartiality 
of regulatory activities: regulators may 
redesign programs to suit industry’s 
wishes because industry is paying the fee. 


11.20 Parliamentarians have expressed 
similar concerns. User charges have been 
discussed in several hearings and reports 
by the agriculture committees of both the 
House of Commons and the Senate. 
Concerns about the impact of user charges 
have intensified with the severe pressures 
on incomes in the sector as a result of low 
commodity prices. 


Focus of the audit 


11.21 Our audit focussed on activities 
surrounding user charges over the past 
five years in Agriculture and Agri-Food 
Canada, the Canadian Food Inspection 
Agency and the Canadian Grain 
Commission. We looked only at their fees 
for services provided to users outside the 
government, and discussion in this chapter 
is limited to such “external user charges”’. 


11.22 In Agriculture and Agri-Food 
Canada, most issues pertaining to user 
charges for intellectual property in the 
Research Branch were distinct from issues 
facing the Department’s other branches 
and the two other organizations. The 


($ Millions) 
60- 


50- 


Canadian Food 
Inspection Agency 


results of our work on the Research 
Branch are reported in Chapter 12 of this 
Report. 


11.23. The Canadian Food Inspection 
Agency began operations in April 1997, 
combining the inspection activities 
previously carried out by three 
organizations — Agriculture and 
Agri-Food Canada, Health Canada and the 
Department of Fisheries and Oceans. Our 
audit examined programs with user 
charges, where relevant, both before and 
after the Agency’s creation. 


11.24 In essence, our audit asked the 
question, “How well are the organizations 
in the agriculture portfolio managing their 
external user charges?” The specific 
objectives of the audit were to determine 
whether the three organizations we 
audited: 


e have properly designed and managed 
user charges so as to achieve the 
government’s objectives for user charges; 


e are learning from experience through 
the regular review of information and 
experience gained in applying user 
charges; and 


e monitor and report performance ina 
way that is clear and meaningful and 
allows Parliament to judge how well user 


Agriculture and 
Agri-Food Canada 


Canadian Grain 
Commission 
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charges have been implemented and 
managed. 


11.25. The government’s policy 
provided a framework for our audit 
criteria on implementing user charges. We 
built on the policy by reviewing best 


practices and literature from other sources, 


including the Organization for Economic 
Co-operation and Development (OECD). 
The key management practices required 
by the government’s policy are broadly 
consistent with the international literature 
on user charges. 


11.26 Details of our audit objectives, 
scope and criteria are at the end of the 
chapter in About the Audit. 
Recommendations for corrective action by 
the three organizations are presented on 
pages 11—34 and 11-35. An insert on 
pages 11—27 to 11-32 summarizes our 
findings on some key elements of 
managing user charges, and notes some 
good practices that might be useful to 
other organizations in the portfolio or 
elsewhere in government. 


Observations 


Implementation of Government 
Policy Is Incomplete 


11.27 Although the three organizations 
we audited have put user charges in place, 
they have yet to fully achieve the 
expectations set out in the government’s 
1997 policy on external user charges. The 
majority of changes to the organizations’ 
user charges predate the 1997 policy. Our 
audit presents observations and 
recommendations in areas where their 
management of user charges needs 
improvement to meet the expectations set 
out in the 1997 policy. As this section of 
the chapter discusses, those areas include 
costing practices, service standards, 
consultations and reporting of results. 
11.28 We recognize that the 
organizations we audited are very 
different from one another in the type of 
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services they provide and the progress 
they have made in implementing the 
government’s policy on user charges. 
While implementing user charges is an 
ongoing task, each organization is at a 
point where it needs to make key 
decisions on user charges. The current 
status and the decisions facing each 
organization are summarized in the insert 
on pages 11-10 to 11-13. 


Improvements needed in program 
costing 


11.29 The Treasury Board Secretariat 
and our Office alike have often 
emphasized the importance of good 
costing systems. The appropriate charge 
for a service can be determined only when 
the full cost of providing the service is 
known. Proper costing can help program 
administrators deliver their programs 
more efficiently and make better-informed 
choices about whether and how best to 
undertake new services or change existing 
programs. We expected that the 
organizations would know the full costs of 
providing their services and would have a 
reasonable understanding of the different 
factors that could affect their costs. 


The three 
organizations have 
shown little 
improvement in the 
way they obtain and 
use cost information. 


11.30 Over the last five years, the three 
organizations we audited have shown little 
improvement in the way they obtain and 
use cost information. They can roughly 
estimate the direct and overhead costs of 
delivering their programs, but have not 
calculated the full costs of the various 
services in each program. We found a 
reasonable costing system in only one 
program — the administration of 
Agriculture and Agri-Food Canada’s Net 
Income Stabilization Account. 


Service standards are not widely used 


11.31 Where user charges are applied, 
we expected that the organizations would 
have service standards. With few 
exceptions, our examination found that 
service standards are not used widely in 
programs with user charges. One — the 
Net Income Stabilization Account — does 


Service standards are 
not widely used. 


(continued on page I]—13) 
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Status and Decisions Facing Each Organization 


ae 
BCC 


The Canadian Grain Commission has two 
major services — weighing and inspecting grain 
destined for export. Its primary purpose is to 
regulate grain handling and to establish and 
maintain standards for quality. 


The Canadian Grain Commission has 
recovered the majority of its costs for decades, 
and close to its full costs for more than 10 
years. The grain industry accepts that the 
regulated inspection and weighing services 
provided by the Commission are a private 
benefit to the grain industry, and that these 
costs are to be recovered fully. The portion of 
the Commission's business that is not 
considered to benefit solely the grain industry 
(mainly the Grain Research Laboratory) 
represents approximately 20 percent of the 
Commission's expenditures, half of which is 
recovered from the industry. 


The existing fee structure was designed to 
recover all corporate overhead costs from the 


Canadian Grain Commission 


fees charged on grain shipments exported 
through seaports. Since then, however, 
shipping patterns have changed. Notably, in 
recent years more grain is being shipped from 
the Prairie provinces directly to the United 
States and not through the seaports. Some staff 
of the Commission were redeployed to provide 
services at primary elevators. The 
Commission’s costs of providing the services 
remained relatively constant. However, the shift 
in shipping patterns meant that revenues 
decreased and, as a result, corporate overhead 
costs were no longer fully recovered; the fee 
structure was no longer aligned with the cost 
base. If fees had been adjusted more quickly, 
the Commission might have been able to avoid 
part or all of its deficit for the 1998-99 fiscal 
year, estimated at more than $10 million. 


The Canadian Grain Commission has 
regularly reviewed its programs and its cost 
recovery about every five years. The most 
recent review came at a time when the 


Commission was confronting the fundamental 
changes in the grain industry and their resulting 
financial pressures. A major emphasis of the 
Commission’s recent review was to better align 
its user charges with its services and those who 
benefit from them. 


The Commission’s review of its programs 
has gone a long way toward implementing the 
government's policy on user charges; however, 
action is yet to be taken to implement the 
review’s recommendations. The 
recommendations include making short-term 
changes to operations and regulations in order 
to reduce the financial pressures on the 
Commission, and longer-term legislative and 
funding changes to give it the flexibility required 
to meet the changing needs of the grain 
industry. Whether it acts on the 
recommendations or not, we believe the 
Canadian Grain Commission can make 
additional improvements, especially in its 
costing of services. 


Cost recovery at the ($ millions) 

; Canadian Grain Commission 1994-95 1995-96 1996-97 1997-98 
Service fees 54.5 45.4 41.7 50.1 
Other _ : 2.1 1.4 ies Wee 
Total 56.6 46.8 43.0 51.3 


yo 
| AciA | Canadian Food Inspection Agency 
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The Canadian Food Inspection Agency 
delivers inspection programs for food safety and 
animal and plant health. These programs are 
supported by laboratory and analytical services, 
and facilitate access for Canadian products to 
markets in other countries. Inspections are 
conducted across the country, at slaughter and 
processing facilities, warehouses, ports of entry 
and farms. Staff of the Agency also conduct 
inspections in other countries for enterprises 
that wish to import animals, plants or goods into 
Canada. 


The major user charge initiative under the 
Canadian Food Inspection Agency began 
before the government-wide Program Review 
of 1994-95, while the majority of the services 
the Agency now provides were still under the 
Food Production and Inspection Branch of 
Agriculture and Agri-Food Canada. Referred to 
as the “Business Alignment Plan”, the initiative 
set out to implement a comprehensive set of 
user charges consistently across commodity 
lines. 


Targets were set to recover a percentage 
of costs for each functional activity, such as 
import or export inspection, based on an initial 
assessment of the proportion of private benefit 
received by users of the service. For example, 
the cost recovery target for inspection of 
imported fresh fruits was set at 50 percent, on 
the assumption that the inspection service 
provides the importers with the opportunity to 
sell their product in Canada while also 
protecting Canadians from the entry of 
potentially harmful pests and diseases. 
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The Agency has partially completed its 
initiative. Fees are in place, but the percentage 
of costs recovered is not consistent across 
commodity lines for similar services even 
though the cost recovery target is the same. 
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This means that different commodity groups 
pay a different proportion of the costs for similar 
services. If there are reasons for the 
differences, such as the competitive nature of 
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the industry, these have not been explained to 
the public. However, Agency officials 
acknowledged that the differences do indeed 
represent inconsistencies. 


Cost Recovery Rates Vary Significantly Across Commodity Groups 


Functional Area - Imports 


Meat hygiene 

Fresh fruit and vegetables 
Animal disease and pest control 
Processed fruit and vegetables 
Dairy 

Processed egg products 

Plant disease and pest control 
Seed 


Shell eggs 


Functional Area —- Exports 


| — 
| [ _| 9% Costs Recovered 
| for 1997-98 

l 


Cost Recovery Target 50% 


Processed egg products 
Fresh fruit and vegetables 
Meat hygiene 
Dee ial % Costs Recovered 
+ —_______ for 1997-98 
Plant disease and pest control - 
Sized" [En a a) Cost Recovery Target 35% 
Processed fruit and vegetables [7 
Seed potatoes 
Animal disease and pest control 
| i} | i] ! i] i] i} i} 
0 10 20 30 40 50 60 70 80 


Note: Commodity groups not shown above have no user charges for these functional areas. 


Source: Canadian Food Inspection Agency 


The Department's Food Production and 
Inspection Branch had very little time to 
implement its fee structure, as the funding voted 
by Parliament had assumed that user charges 
would be in place and would generate the 
planned revenue. This fiscal pressure continued 
after the Agency's creation, since it was 
expected to make the delivery of inspection 
services less costly to the federal government. 
Revenue from user charges was needed 
immediately, and identifying areas to reduce or 


avoid costs had secondary priority. At a higher 
level, however, the Agency's creation was itself 
designed as a form of cost reduction. 


During its consultations with industry 
groups on user charges, the Food Production 
and Inspection Branch had been open about its 
immediate need to generate revenue and about 
the principles on which user charges would be 
based. The Agency subsequently set revenue 
targets for each commodity group. After 


presenting its initial proposals for fee structures, 
the Agency generally accepted 
counterproposals from industry as long as this 
did not jeopardize the overall revenue target. 
Accepting these additional proposals was 
partially responsible for increasing the number 
of individual fees to more than 1,600. The 
Agency recognizes that it is costly to administer 
this many fees and the associated regulations, 
and it hopes to streamline them in the future. 
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While the Agency pursued revenue 
generation vigorously, it also responded to 
industry's demands for new or more efficient 
ways of delivering services and it sought 
opportunities to avoid and reduce costs. 
Livestock grading is a prime example of cost 
avoidance that resulted in net savings to the 
government and industry. 
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The Canadian Food Inspection Agency 
manages one of the government’s most 
complex sets of user charges. Previously, the 
Agency had established targets for recovering a 
percentage of inspection costs in functional 
areas such as import, export and domestic 
inspection. But it has yet to meet those targets 
for all commodities. Much of the Agency's focus 


in its first two years was to make the necessary 
organizational and program changes to bring 
together staff and programs from four different 
departments. Senior managers believe that 
these changes are largely in place, and they are 
now free to renew their concentration on user 
charges. 


Cost recovery at the ——— —= ($ millions) E = 

Canadian Food Inspection Agency 1994-95 1995-96 1996-97 1997-98 
Animal health 1.0 Sh 45 47 
Fresh fruit and vegetables 2.6 48) 3.9 48 
Livestock and poultry grading 3.0 ChYA 1.9 1.3 
Plant health le 2.4 3.1 cota) 
Seeds 1129) 1.6 2.5 7-25) 
Meat hygiene | Shi 13.5 17.0 16.7 
Other (dairy, feed, processed products, etc.) 4 5) 2 2 
Fish inspection* 8 AY) P13 6.1 
Total 14.1 28.2 36.6 41.9 


Pars 
| Cael Agriculture and Agri-Food Canada 


For the most part, each branch of 
Agriculture and Agri-Food Canada that we 
audited (Policy Branch, Market and Industry 
Services Branch and Prairie Farm 
Rehabilitation Administration) manages its own 
user charges. Each branch evolved separately 
and has a distinct history and line of business. 


The Prairie Farm Rehabilitation 
Administration has been involved in cost 
recovery since 1939 (see Chapter 24 of our 
1997 Report). User charges associated with its 
Community Pastures program are well 
developed. They compare with market rates 
and recover more than 80 percent of the 
program's direct costs. The remainder of the 
costs are attributed to the “public good” 
component — maintaining the pastures in an 
environmentally sustainable manner. The 
Prairie Farm Rehabilitation Administration is 
working toward recovering a greater 
percentage of the costs of irrigation services it 
provides in southwest Saskatchewan. 


Costing is one area where the Prairie Farm 
Rehabilitation Administration still needs to 
improve, as it does not yet collect and allocate 


* Prior to 1 April 1997, fish inspection was carried out by the Department of Fisheries and Oceans. 


costs and overhead for each of its current and 
prospective services and programs. 


The Net Income Stabilization Account 
administered by the Policy Branch represents 
the Department's greatest progress toward fully 
implementing the government's policy on user 
charges. It has developed a solid costing 
methodology (activity-based costing) that it can 
use to cost out possible changes to the 
program. The Net Income Stabilization Account 
has also shown evidence of good consultation 
with stakeholders, and has integrated service 
standards and measures of client satisfaction 
into its management culture. In 1995 the Net 
Income Stabilization Account announced to its 
national advisory committee that it intended to 
move toward full cost recovery. 


The Market and Industry Services Branch 
has only one user charge, in connection with 
the Farm Improvement and Marketing 
Cooperatives Loans Act. This is a loan 
guarantee program that provides producers and 
their co-operatives with access through financial 
institutions to intermediate term credit at 
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reasonable rates. The recent fee increase in 
1999 was designed to recover the full costs of 
administration and loan defaults under the 
program. ; 


It is not clear why certain programs recover 
full costs, others recover partial costs and still 
others recover nothing. For example, the Net 
Income Stabilization Account program recovers 
approximately 70 percent of its administration 
costs, while the Agriculture Income Disaster 
Assistance program does not. If program 
considerations suggest that something less than 
full costs should be recovered, those 
considerations need to be made clear to 
parliamentarians and stakeholders. Currently 
there is no set of guiding principles to ensure 
that user charges are implemented in a 
consistent way throughout the Department. 


In other aspects of managing user 
charges, Agriculture and Agri-Food Canada 
uses reasonable consultation practices and has 
simplified fee collection, but needs to look for 
opportunities to reduce the length of time it 
takes to introduce or amend fees. 
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Cooperatives Loans 


Research Branch 


Total 


(continued from page 11-9) 


Canadian Grain Commission has 


11.32 


($ millions) 
Agriculture and Agri-Food Canada 1994-95 1995-96 | 1996-97 | 1997-98 
Farm Improvement and Marketing —— | : : 
2.8 1.9 2.4 2.6 
Net Income Stabilization Account 5.5 5.4 76 8.2 
Prairie Farm Rehabilitation Administration 11.8 12.7 13.0 | 13.0 
15: ao 3.4 | 3.4 
21.6 22.5 26.1 | 27.2 
simply a reaction to having to pay for 
something that used to be “free”, a study 
routinely measure the services it provides performed by Agriculture and Agri-Food 
against predetermined standards. The Canada does suggest that on certain 
industries the impact has been greater (see 
developed service standards but has not Exhibit 11.2). 
reported whether they have been met. The 
Canadian Food Inspection Agency drafted 11-34 In accordance with the 
service standards but has implemented government's policy, we expected that the 
them in only two main areas: the feeds designers of user charges would use the 
program applies standards and compares information gained from impact 
performance against them, while fish assessments, including the potential 
inspection has adopted standards but does !Mpact on competitiveness. The federal 
not compare performance against them. government’s Business Impact Test is a 
tool that can assist in these assessments. 
Clients want to know the level of We found that the organizations in our 
service they will receive for the fees they audit do not use such tools on a regular Impact assessments 
pay, and they want to know that they will basis. were imprecise and 
receive full value for their money. In a TAS Ce oe eae and 


competitive market, the customer can 
choose the supplier who offers the best 
price, quality and service. In a regulated 
environment, where there may be only 
one supplier, the customer has no such 
choice. In those cases, communicating 
service standards and reporting 
performance against them serve as a form 
of accountability to the public and users of 
the service that helps to compensate for 
the lack of choice of suppliers. 


Are the impacts of user charges well 
understood? 


11.33. Clients and stakeholders have had 
much to say about the impact of user 
charges on their businesses and livelihood. 
Although some of their comments may be 


lacked quantification. 
enforced through regulations. In 


accordance with the government’s 
regulatory policy, we expected that the 
intended benefits of new fees or fee 
changes would be compared with the 
expected cost of implementing them. 
However, the assessments did not 
routinely identify or include benefits, 
especially indirect or societal benefits. 
Where such non-monetary benefits have 
been anticipated, they have not been well 
documented or communicated to 
stakeholders or parliamentarians. These 
benefits and costs — including the costs 
of implementing and administering the 
fees — need to be assessed. 


11.36 The impact assessments of 
proposed regulatory amendments that we 
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reviewed were imprecise and lacked 
quantification. Further, we had difficulty 
finding evidence in the assessments to 
support their frequent contention that the 
fees would have little or no effect on 
competitiveness because they were such a 
small percentage of the costs of the 
products inspected. The impact 
assessments contained little information 
that would help the reader understand the 
fees’ expected impacts. 


11.37. While individually each fee may 
seem insignificant, in total they can 
amount to a significant portion of net 
income in some industry sectors. The 
revised government policy states that 
departments and agencies must “work 


with clients to assess the cumulative 
impact of multiple fees from all federal 
sources, and assess proposed fees in that 
context.” Agriculture and Agri-Food 
Canada has analyzed the cumulative 
impact of the user charges in its portfolio 
(see Exhibit 11.3). 


Exploring alternatives before 
establishing user charges 


11.38 There may be alternatives to 
imposing user charges, and these need to 
be explored before the charges are put in 
place. Clients do not want revenue from 
user charges to pay for inefficiencies in 
the way programs are delivered. We 
expected that the organizations would 
analyze and consider options for reducing 


Exhibit 11.2 


Cumulative Effect of Federal User Charges Varies by Sector 


Dairy, 
Grain and poultry 
oilseed Potato Other and egg Hog Cattle Red meat Other 
%o farms farms farms farms farms farms processing _ processing 
0.0 _ 
-0.5 
—1.0 
-1.5 
—2.0 
—2.5 
-3.0 
—3.5 


Source: Impact of Selected Federal Cost Recovery Initiatives on the 
Agri-Food Sector, Agriculture and Agri-Food Canada 
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or avoiding costs, recognizing that 
subsequent consultations with users might 
identify additional options, and that they 
would be able to demonstrate that 
activities are delivered efficiently. 
Avoiding costs means “getting out of the 
business” or taking measures to avoid cost 
increases that would otherwise be 
incurred. Reducing costs requires that 
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service levels be lowered or else delivered 
more efficiently. An example of cost 
avoidance as an alternative to additional 
user fees is described in the case study on 
page 11—23, Cost Recovery: Benefits 
Achieved Beyond Revenue Generation. 


11.39 Assessing alternatives includes 
determining which alternative best meets 


Exhibit 11.3 


A Significant Advance in Understanding Cumulative Fee Impacts 


In the past, very little work was done to determine the combined impact on individuals and companies of the various user fees they 
pay. Parliamentary committees, stakeholders and the Treasury Board Secretariat have called on federal organizations to study the 
cumulative impact of federal user fees on specific sectors. 


Some observers argue that concern about fee impacts on industry is misplaced — that as long as the fee is appropriate and industry is 
benefiting from the government service, then user fees are a legitimate cost no different from other costs of doing business. In our 
view, however, analysis of fee impacts is important because it helps decision makers to make informed decisions about user fee 
policy — regulators must ensure that the combined impact of user fees does not jeopardize program objectives. 


In 1998, Agriculture and Agri-Food Canada was the first department to undertake a cumulative impact study of federal fees. The 
study has been of considerable interest to the agriculture sector, and was the subject of a hearing by the House of Commons Standing 
Committee on Agriculture. The study examined the impact of fees on the agriculture and agri-food sector, using economic modelling 
to identify how fees are passed on to customers and suppliers. The study concluded that impacts vary among different industries 
within the sector, reducing net operating income anywhere from virtually zero to 3.4 percent. 


Lessons were learned for future studies 


The experience gained from conducting the study highlighted a number of lessons for similar future studies: 


e Several aspects of the project design proved particularly effective. Other fee-charging federal organizations and industry 
stakeholders were extensively involved in decisions on the project design. Macro-level economic analysis was supplemented by a 
micro-level analysis of potential impacts on illustrative farms and food-processing companies, which helped identify the 
differential effect of fees on different users. 


e The analytical methodology was based on reasonable economic principles adapted from recent studies by the Department and by 
the Organization for Economic Co-operation and Development. The methodology, however, was not explained well in the report, 
nor was the analysis documented adequately. This made it impractical for us to verify the calculations, and could hamper future 
studies that attempt to build on the approach. 


e A sensitivity analysis would have shown how sensitive the findings were to changes in assumptions and design parameters and 
thereby helped to build a better understanding of the reliability of the study’s results. 


e Obtaining good data from other federal departments and clarifying data definitions was a time-consuming process. Standard data 
and definitions across government would greatly facilitate future cumulative studies. 


More needs to be known about fee impacts 


The 1998 Agriculture and Agri-Food Canada study significantly advanced knowledge about user fee impacts. However, some 
important questions were not addressed, given that the study scope was already ambitious. No policy analysis was conducted of what 
changes, if any, were warranted in user fee policy in light of the study’s results. Future studies in the agriculture and agri-food and 
other sectors need to examine whether user charges are a significant factor in international competitiveness, and need to consider 
refining the analytical tools used to capture secondary fee impacts and changes in industry behaviour. 


There is also an important opportunity to convert the study methodology into a tool that would assist in the impact analysis of 
individual proposed fees. Until the study, impact analyses did not examine how fees are passed from the fee payer on to customers 
and suppliers. 
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The framework for 
consultations has 
improved over the past 
five years. 


the objectives of the organization, the 
program and the needs of all stakeholders. 
Good costing systems are needed to assess 
alternatives properly. 


11.40 We found that the organizations 
in our audit have in place or are 
considering a number of new and creative 
ways to deliver mandatory services. On a 
larger scale, the Canadian Food Inspection 
Agency’s creation was itself a major cost 
reduction initiative, designed to reduce 
government spending by $44 million 
without increasing user charge revenues. 
However, we found little evidence to 
suggest that the organizations analyze 
options such as cost avoidance in 
sufficient depth before user charges are 
applied. 


Consultation process has improved 


11.41 We expected that the 
organizations would implement the 
government’s policy that departments 
must undertake “meaningful and 
effective” consultation with clients 
throughout the fee-setting process. 
“Meaningful and effective” means that 
clients believe their concerns have been 
given a fair hearing and that they 
understand the rationale for the fees. 


11.42 In the absence of appropriate 
consultation, there is a risk that opposition 
to a proposed fee could derail a user 
charge initiative and result in lost revenue 
to the Crown. The organization also risks 
charging an inappropriate amount or 
charging the wrong party. A further risk is 
that the organization will miss valuable 
advice from clients on how it can improve 
the service. 


11.43 Clients often mount strong 
opposition if they believe fees are being 
imposed unilaterally. They are more 
accepting when the fees provide an 
agreed-upon level of service that clients 
identify as a private benefit to them. 


11.44 We found that the framework for 
consultations has improved over the past 
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five years. Agriculture and Agri-Food 
Canada and the other organizations have 
made an effort to ensure that consultations 
are properly structured. Large and small 
producers and industries are better 
represented in recent years, although 
smaller producers find it costly to take 
time away from their business for 
consultations. 


11.45 Clients who pay the fees — the 
private beneficiaries of the service — 
have been represented in consultations by 
their national associations. However, apart 
from government officials protecting the 
interests of the public, there has been little 
representation of the beneficiaries who do 
not pay fees. In the consultations held 
during its recent review, the Canadian 
Grain Commission was successful in 
engaging some of the broader 
beneficiaries (producers) beyond those 
who pay the fees (grain handling 
companies). The Canadian Food 
Inspection Agency has attempted to 
engage consumer groups in consultations 
on food issues, but with limited success. 
When representation of the public good is 
relatively low, the results of consultations 
may be perceived to favour industry. 


11.46 Nevertheless, we found that 
consultations were generally open and that 
feedback was provided to the consultation 
groups on a timely basis. The government 
organizations involved have changed fee 
structures or amended fees in response to 
stakeholder concerns, as well as altering 
the way they deliver services. 


11.47. At the same time, however, a 
number of stakeholders who participated 
in consultations with the Canadian Food 
Inspection Agency told us that in their 
view, the Agency had used the meetings 
not so much to consult as to announce its 
planned fee increases. 


11.48 ~=While their consultation 
processes have improved, the three 
organizations could make them more 
effective by using them as a regular, 
ongoing means of sharing information 
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about program performance and future 
objectives. Client satisfaction can be 
improved with proper consultation and 
reporting of results against service 
standards. 

Developing alternative fee structures 


11.49 We expected that organizations 
would consider alternatives to setting a 
uniform fee for all transactions. There is 
an inherent trade-off in choosing between 
a uniform fee with simple administration 
and a more complex fee that costs more to 
administer but better meets the clients’ 
needs and the program objectives. 


11.50 The organizations in our audit did 
attempt, usually during the consultation 
process, to ensure that their fees were 
structured to meet users’ needs. Examples 
from the Canadian Food Inspection 
Agency include lowering fees for smaller 
enterprises and basing charges on the 
length of time it takes to deliver the 
service. 


11.51 While the organizations have 
responded to their clients’ needs, they 
have not adequately considered the costs 
of doing so. The Canadian Food 
Inspection Agency now charges more than 
1,600 fees. Besides the cost of delivering 
the services, there are additional costs to 
maintain regulations and administer 
invoices and fee collection, as well as the 
hidden costs to both staff and the clients 
of the Agency in keeping track of the fees 
and filling out the proper forms. The 
Agency has not estimated these costs, but 
it needs to consider them in choosing one 
fee structure over another. 


11.52. One area where the organizations 
have been able to keep costs low is the 
collection of fees. Some programs have 
been designed to accomplish this, while 
others have improved their collection 
processes. 


Changing fees is a lengthy process 


11.53. We expected that changes to fees 
would be implemented in a timely 
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manner, but we found that the process of 
implementing new fees and fee 
amendments is very slow. It is not 
uncommon for regulatory changes to take 
more than two years. Indeed, recent 
changes to the fee under the Farm 
Improvement and Marketing Cooperatives 
Loans Act program took almost four years 
from the initial proposal until the 
regulation was in place (see the case study 
on page 11-18, The Slow Pace of Change 
Has a Price). 


11.54 Fees are normally established 
following the regulatory process, which 
includes publication in the Canada 
Gazette. The statutory framework 
governing user fees includes the 
Constitution Act, which provides that 
legislation for any tax or charge must 
originate in the House of Commons, 
various other Acts that confer authority to 
establish and amend fees in relation to 
specific activity, and section 19 of the 
Financial Administration Act, which 
permits the Governor in Council to set 
fees or to authorize ministers to set fees by 
order. This regulatory process involves 
various steps that, while protecting the 
public from the indiscriminate and 
arbitrary application of fees, can often be 
lengthy and burdensome. 


The process of 
implementing new fees 
and fee amendments 
is very slow. 


11.55 Streamlining is possible but 
would require better disclosure. Some 
streamlining is possible — the Canadian 
Food Inspection Agency Act allows the 
Minister of Agriculture and Agri-Food to 
set fees after consultation and publication 
of the proposed fee in the Canada Gazette. 
The Agency has used this provision to a 
limited extent for technical changes but 
has not yet used it to change fees, as most 
are related to other legislation. The 
Agency expects that recently tabled 
amendments to the Acts it administers will 
resolve this. 


11.56 Any streamlining would need to 
be balanced, so that parliamentarians and 
external stakeholders would have 

sufficient opportunity — through impact 
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analysis and better reporting — to 
scrutinize user fees adequately. 


11.57 _ Delays in implementing fee 
changes can have a significant financial 
impact on the organization as well as its 
clients, in the form of possible 
underpayment or overpayment of fees in 
the meantime (depending on whether the 


changes proposed were to raise or lower 
the fee). 


11.58 Delays can actually work against 
the interests of users who pay fees. For 
example, in 1997 the Canadian Grain 
Commission initiated a reduction of the 
grain receival service fee. From design to 
implementation, the change took 


The Slow Pace of Change Has a Price 


Administrators of the Farm Improvement 
and Marketing Cooperatives Loans Act 
(FIMCLA) program recently introduced several 
regulatory changes, including an increase in 
user charges and expanded criteria for 
program eligibility. These changes were almost 
four years in the making. 


The chronology below illustrates the long 
road to regulatory change for this latest 


initiative. 


The regulatory process was not solely 


these changes. Three other factors created 
further delays: 


The consultation process was unusually 
long. During this time, additional requests 
for amendments were proposed by 
stakeholders. 


From the inception of the process to its 
conclusion, three different managers were 
responsible for the FIMCLA program. 


The impact of low commodity prices 
worsened in 1998, and was an external 
factor that reasonably could not have been 
planned for or anticipated. 


Consultations with stakeholders begin. Stakeholders want other changes in related regulations. 


This case demonstrates that regulatory 
change does take time and that other factors 
can further slow the process. Officials need to 
anticipate developments and plan well ahead in 
order to ensure that their programs (and any 
related user charges) continue to achieve their 
intended objectives and remain responsive to 
users’ needs. In this case, stakeholders had to 
wait two years to gain access to program 
changes they had agreed to back in April 1997. 
As well, approximately $3.5 million in additional 
registration fees were lost to the Crown from 
April 1997 to May 1999 as a result of these 
delays. 


Feb. | Cabinet approves increase in maximum allowable FIMCLA loans from $1.5 billion to $3 billion and approves increase in loan 
guarantee fee from 0.50% of the principal amount of the loan to 0.75%. 


NEGOTIATIONS WITH STAKEHOLDERS 


Apr. Stakeholder agreement reached on issues raised in consultative process. 


Revised draft regulations for FIMCLA finalized. The Treasury Board Secretariat approves a further increase of 0.10% in 
loan guarantee fee to replace funding for cuts in administrative costs during 1995 Program Review. 


responsible for the slow progress in making 

1 

9 =e 

9 June — FIMCLA amended (15 June 1995). 
5 

1 

9 

9 

6 

1 

9 |» 

9 | July 

Ul 

1 

9 June — Proposed regulatory changes published in Canada Gazette — Part |. 

et 

: Sept. Proposed regulatory changes sent to Minister for approval. 
1 

9 ———— a 

9 

9 | 
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Mar. Minister approves publication of proposed new regulations in the Canada Gazette — Part II. 


May New regulations come into effect. Loan registration fee is now 0.85% of principal value of FIMCLA loan. 


14 months — 4 months longer than the 
Commission had anticipated. When fees 
are being reduced with the agreement of 
all stakeholders, a more streamlined 
process would benefit everyone. 


11.59 Fee structures inhibit 
responsiveness. As a delay in a fee 
change becomes extended, there is a risk 
that revenues will no longer offset 
expenses as intended, thereby increasing 
financial risk to the organization. This risk 
is more significant for the Canadian Grain 
Commission than many government 
entities that charge user fees, as it relies 
almost entirely on fees for its funding. In 
addition, the Commission has little control 
over the time it takes to change fees; this 
is dictated by the regulatory process. A 
streamlined regulatory process is needed, 
or else a more dynamic way of adjusting 
fees in response to changes in costs. 


User charges are not well integrated 
into strategic planning 


11.60 We expected that the 
organizations would have a 
comprehensive and consistently applied 


Integrating User Charges Into an Organization’s Strategic Planning 


In 1998, the Canadian Grain Commission 
began a comprehensive review of its programs. 
The objective was to confirm the Commission’s 
responsibilities and to update programs to 
ensure that its services would remain effective, 
economical and efficient. The review resulted in 
a report recommending changes to the way 
services are provided and funded, including 
user charges. 


The process followed by the review shows 
how an organization can integrate user 
charges into its strategic planning: 


e lt was comprehensive: each service offered 
by the Canadian Grain Commission was 
reviewed. It examined in depth a number of 
critical questions, including whether or not to 
retain a program outside the Commission’s 
core mandate, and the potential savings and 
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strategy for user charges, with clear, 
achievable objectives that promote the 
efficient allocation of resources. 
Moreover, the Treasury Board expected 
organizations to include a strategy for cost 
recovery in their 1998—99 business plans. 


11.61 A user charge strategy is intended 
to identify activities where the 
organization could introduce user charges, 
and areas where overriding policy 
objectives would take precedence. 


11.62 Of the organizations we 
examined, only the Canadian Grain 
Commission has communicated a user 
charge strategy to parliamentarians and 
stakeholders as part of its 1998-99 review 
of programs (see case study on this page, 
Integrating User Charges Into an 
Organization’s Strategic Planning). 


11.63 Beyond having a user charge 
strategy, an organization needs to integrate 


user charges into its key decisions at the 
program and organizational levels and 
make it part of its management culture. 
User charges are a management tool — 


one of many used to achieve an 
organization’s objectives. Our 


impacts if the program were to be 
terminated; how to deliver retained programs 
more efficiently or at lower cost; the 
appropriate approach to recover the cost of 
the programs, and the revenue target for 
each; the priorities for allocating resources; 
whether there were new programs or 
services with revenue-generating potential 
that would fit with the Commission’s core 
mandate; and whether changes to the 
Canada Grain Act were needed. 


It considered fundamental changes such as 
cost reduction, cost avoidance and 
alternative fee structures. 


It used staff from other sections of the 
organization to reduce bias. (A review team 
was established for each group of services, 
headed by senior staff from outside that 
group of services.) 
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e Consultations were extensive and well 
managed. A stakeholder advisory board was 
established that included representation from 
all stakeholders. Cross-country consultations 
were also held. 


e Senior management buy-in was perceived as 
sincere. Both staff and stakeholders told us 
that senior management supported the 
review process. 


The success of the review may not be 
known for some time. The Commission has 
received authority to implement fee changes in 
its optional services, but in its regulated 
services this must wait until the end of the 
moratorium on user charges. However, the 
review process itself serves as an example of a 
good management practice. 
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Without a user charge 
strategy, user charges 
may be applied 
inconsistently among 
programs. 
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examination found that the organizations 
present little information on user charges 
in their business plans, beyond amounts 
shown as forecast revenue. 


Planning of user charges could be 
improved at the program level 


11.64 To ensure that their strategic 
plans are reflected in their programs, 
organizations normally use planning 
frameworks that produce a strategic plan, 
a business plan and individual program 
plans. We found little evidence that the 
organizations in our audit consider user 
charges in any planning at the program 
level except to plan specific fee 
initiatives. If a strategy for user charges is 
not linked clearly from the strategic plan 
to individual program plans, user charges 
might be omitted from the design of new 
programs or applied inconsistently among 
existing programs. This can result in 
inefficiency or in failure to achieve the 
strategic objectives. 


11.65 Organizations have included user 
charges in their planning in response to 
particular circumstances, but they need to 
do it on a regular, ongoing basis. For 
example, the Canadian Food Inspection 
Agency has not updated its user charge 
plan regularly, although a comprehensive 
plan was used when the cost recovery 
initiative was introduced in 1994. 


11.66 Horizontal issues such as user 
charges need to be addressed in a similar 
way throughout the organization. We 
noted that, in the past two years, 
Agriculture and Agri-Food Canada has 
done a lot of work to establish a 
Performance Framework that sets out how 
the Department plans to meet its 
objectives. The Framework addresses 
several cross-cutting issues, such as 
integrating environmental considerations 
into decision making, but does not 
mention user charges and the efficient 
delivery of programs. The Framework or 
some other vehicle could be used to 


ensure that the Department’s objectives 
for user charges are achieved consistently. 


11.67 Planning for user charges at the 
program level usually concentrates on 
specific fee initiatives. Not considering 
user fees in annual planning at the 
program level means that opportunities 
can be missed to implement new fees or 
reduce existing fees. 


Guiding principles for user charges can 
help 


11.68 An important part of developing 
a user charge strategy is articulating the 
principles that support it and that will 
guide its implementation. Each of the 
organizations has various managers 
responsible for designing or modifying 
programs. In order to ensure that the 
strategy for user charges is delivered 
consistently throughout their organization, 
managers need guidance on how to 
interpret the government’s policy and 
apply it to their program’s particular 
circumstances. 


11.69 —_ User groups also benefit from 
guiding principles, which can serve as 
“rules of the game”. They provide 
important information to stakeholders, 
giving them a framework to judge whether 
existing or proposed user charges are fair 
and appropriate. Clearly articulated and 
publicly disclosed principles can 
encourage the acceptance of user charges 
among user groups and deflect attempts 
by certain user groups to avoid them. 


11.70 The Canadian Food Inspection 
Agency’s “General Principles for Cost 
Sharing” were established in 

February 1994, while the inspection 
services were still part of the Food 
Production and Inspection Branch of 
Agriculture and Agri-Food Canada. Some 
of the broad principles, similar to the 
more recent government policy on user 
charges, stipulated that: 


e the private individual or enterprise 
who used a service or received an 
identifiable economic benefit from the 
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Branch was expected to pay the 
government’s costs of providing that 
service; 


e fees should not be viewed as a source 
of general government revenue; 


e consistency across commodities, 
programs and functions should be an 
objective; 


e fees charged should not exceed the 
cost of providing the service; 


e the development and implementation 
of fees should be a transparent and open 
process; and 


e fees should not affect the quality, 
safety or reliability of agri-food products, 
and should not interfere with international 
trading requirements. 


11.71 The Food Production and 
Inspection Branch had also established 
guides or targets for determining the 
relative proportion of private benefit and 
public good in each functional area, which 
the Canadian Food Inspection Agency 
continues to support. It was assumed that 
there was an element of private benefit in 
all the services provided by the Branch 
unless it determined otherwise. For the 
private benefit portion, targets 
(percentages to be recovered) were set for 
functions such as import and export 
inspections, registration of establishments, 
labelling, and overtime. 


11.72 Together, these principles and 
targets represented a positive step and 
helped gain faster acceptance of new fees 
and fee increases from 1994 to 1998. 
However, the principles have not been 
updated since the creation of the Canadian 
Food Inspection Agency. Groups from 
each of the three originating departments 
brought their own practices and 
experience to the Agency; it is important 
that the Agency adopt the best of these in 
managing its user charges and services. 


11.73 We found that Agriculture and 
Agri-Food Canada lacks guiding 
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principles (see case study on page 11-22, 
Similar Programs — Dissimilar User 
Charges). The Canadian Grain 
Commission has had guiding principles 
for its user charges since 1994, 


Determining private benefit and public 
good can be difficult 


11.74 One of the most difficult aspects 
of implementing user charges for a service 
is determining what proportion of the 
benefit is received respectively by the 
users and by the general public. The 
decision can significantly affect the size of 
the fee that users will pay. 


11.75 We expected that each of the 
organizations we audited would have 
established criteria on which to base the 
allocation of private benefit and would 
have clearly communicated those criteria 
to stakeholders. Determining the relative 
proportion of public good to private 
benefit is a highly judgmental process, 
and it is important that the people who 
pay the fees understand and accept the 
way the charges are formulated. 


11.76 As we have noted, the Canadian 
Food Inspection Agency used a “rule of 
thumb” to determine the amount of 
private benefit in the services it provides. 
With few exceptions, it set cost recovery 
targets ranging from a low of 10 percent 
for some activities to almost full cost 
recovery for services it judged to provide 
an entirely private benefit. 


11.77 ~— As noted in our 1996 Report 
Chapter 9, Animal and Plant Health: 
Inspection and Regulations, the 
Department (and later the Agency) was 
committed to developing criteria for use 
in estimating the private benefit associated 
with each functional area of service; the 
Agency has not yet done this. Likewise, 
Agriculture and Agri-Food Canada and 
the Canadian Grain Commission have not 
developed or explained how they identify 
and value the public and private benefits 
of a service. 
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Similar Programs — Dissimilar User Charges 


How would guiding principles help to 
clarify important decisions about user 
charges and promote consistency in an 
organization? 


similarities and both confer private benefits on 
program participants. Yet the user charges for 
the two programs are at opposite ends of the 
spectrum. The Department has no user charge 


considerations. However, it is important that the 
Department be clear about how it determines 
when user charges should apply, and then 
disclose, where appropriate, the nature of policy 


principles or other publicly available information 
that explains the rationale for these differences. 


considerations or other factors that played a 
role in determining the amount to be charged. A 
set of clear guiding principles would help users 
understand such differences and would also 
help ensure consistency across the 
Department. 


Consider two programs operated by the 
Marketing and Industry Services Branch of 
Agriculture and Agri-Food Canada — the Farm 
Improvement and Marketing Cooperatives 
Loans Act (FIMCLA) program and the Cash 
Advances program. Both programs share many 


The inconsistent treatment of user charges 
in these two programs may have very good 
reasons, such as overriding policy 


Farm Improvement and Marketing 


Cooperatives Loans Act Program Cash Advances Program 


Nature of program Loan guarantee Loan guarantee 


Maximum federal liability 95% of amount guaranteed 85% to 99% of amount advanced, depending on 


default history of producer organization 


Make short-term cash advances available to 
producers of storable crops at harvest time, allowing 
| them to hold the product until later in the year when 
| market prices may improve 


Program objective Increase the availability of medium-term loans for 
improving and developing farms and for processing, 
distributing or marketing of farm products by 


co-operative associations 


Private benefits e Increased returns to producers (long-term) | e Increased returns to producers (in current crop 


e Enhanced availability of medium-term credit year) 


e ebedieca inal e Interest-free loan up to $50,000 


_ Reduced interest rate for advances above 


$50,000 
Averageloan $22,000 $20,000 
Main program delivery agent © Commercial lending institutions Producer organizations 


Main federal costs e Program administration e Program administration 
Defaults 


e Interest on advance of first $50,000 


Delivery agents have user YES YES 
charges? 
Federal user charges permitted YES YES 


by enabling legislation? 


Federal user charges 


as a % of full costs 100% 


; 
i 
j 
| 
| 
| 
i 
e Defaults | ° 
| 
| 
| 
| 
| 
| 
| 
i 
| 


NIL - no federal user charges 
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User charges can provide more than 
just revenue 


11.78 Non-monetary benefits can be 
achieved if fees are designed to capture 
them. The organizations are achieving 
them on a limited basis, but their practices 
could be expanded (see case study on this 
page, Cost Recovery: Benefits Achieved 
Beyond Revenue Generation). As noted 
previously, food inspectors often charge 
the same amount for inspections that have 
found deficiencies as for those that have 
not, even though finding problems entails 
additional inspection time. This provides 
no incentive for the industry to comply 
with regulations and thereby minimize the 
amount of time government inspectors 
need to spend. 
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11.79 We found evidence that the 
design of some user charges has worked 
against the desired behaviour. For 
example, fish processors require an export 
certificate in order to export their product. 
If there is a problem with the product, its 
packaging or labelling, and no export 
certificate is issued, a fee is not charged. 
In order to receive the certificate, the 
processor must correct the problem and 
request another inspection. It is possible 
that inspectors could inspect the same 
product three or more times but charge a 
fee only when they issue the certificate. If 
the product continues to fail the inspection 
and no certificate is issued, the processor 
is not charged for any of the inspections. 


11.80 In summary, the Canadian Food 
Inspection Agency has considerable 


Cost Recovery: Benefits Achieved Beyond Revenue Generation 


The Treasury Board policy emphasizes 
that user charges can bring benefits beyond 
the revenue they generate. The case of 
livestock grading services demonstrates the 
broader benefits that are possible. 


User charges for federal livestock grading 
services were introduced in 1985. Even though 
fees were low at $0.20 per beef carcass and 
$0.09 per hog carcass, the demand for grading 
services declined significantly. This 
demonstrated one of the benefits of cost 
recovery — users who must pay for a service 
decide quickly how much service they really 
need. 


In 1994, the beef industry and the hog 
industry were notified that because the 
government viewed grading to be a 
100 percent private benefit to industry, user 
charges for grading services would increase 
annually to reach full cost recovery by 2000. 
This was based on the judgment that grading is 
an assessment of the quality of the product and 
not of food safety. In the face of full cost 
recovery, with charges estimated to be about 
$1.40 to $1.44 per beef carcass and 


approximately $0.30 per hog, the beef industry 
decided to form the Canadian Beef Grading 
Agency (CBGA), a private non-profit 
corporation. The beef industry was convinced 
that it could maintain standards of accuracy 
and integrity at a much lower cost. The hog 
industry decided that it no longer needed 
federal grading standards because hog grading 
was used only as a mechanism of settlement 
between producers and packers. 


Federal beef-grading regulations remain in 
force. The role of the Canadian Food 
Inspection Agency in beef grading is now 
limited to a national audit role, as part of its 
ongoing accreditation of the Canadian Beef 
Grading Agency. The Canadian Food 
Inspection Agency also assigns a federal 
designation to graders, based on training 
provided by the CBGA. 


Benefits Achieved 


The Canadian Beef Grading Agency has 
been able to lower grading fees three times in 
the three years it has been in operation. It has 
been able to do this by significantly reducing 
the number of graders it needs to carry out its 
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services and by reducing corporate overhead. 
For example, one large plant that had nine 
graders or grader/inspectors before the CBGA 
began now has only two graders working full 
time and one part time. The average fee it 
charges to industry for beef grading has been 
reduced to less than $1 per carcass. 


Because grading was not linked to its core 
services for food safety, the Canadian Food 
Inspection Agency was able to devolve the 
grading function and thus avoid the associated 
costs. Between 1995 and 1997, it was able to 
reduce all but one of its staff assigned to beef 
and hog grading — a total of 87 full-time 
equivalents. 


Benefits Need to Be Identified 
Systematically 


There is a widespread view that user 
charges are no more than a way to generate 
revenue — simply a tax by another name. 
Clearly, there are cases where broader benefits 
have been achieved. The audited organizations 
need to evaluate the extent of these various 
benefits across their programs and to identify 
and publicize user charge success stories. 
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The organizations 
have not developed 
measures of program 
efficiency. 


11-24 


opportunity for using its fees to help 
achieve its program objective of fostering 
industry compliance and, at the same 
time, to better reflect the costs of 
providing the service. 


Little review of performance 


11.81 We expected that the 
organizations would measure their 
performance to determine, among other 
things, whether they were delivering 
programs efficiently. The Prairie Farm 
Rehabilitation Administration uses direct 
feedback from clients in a limited way to 
measure its performance in individual 
projects. Except for the Net Income 
Stabilization Account, which reports 
budget levels, business objectives and 
results to its national advisory committee, 
none of the other fee-charging programs 
we examined have developed measures of 
program efficiency. As a result, they are 
not in a position to demonstrate to clients 
that the programs are being delivered 
efficiently and that fee levels do not 
reflect the costs of inefficiency. 


11.82 Possible measures of efficiency 
could include: 


e benchmarking against comparable 
organizations; 


e comparison with similar activities 
within the organization; 


e comparison with past performance; 
and 


e in-depth reviews of specific 
programs. 


11.83 Program delivery needs to be 
monitored. Another reason to review 
program performance is to make sure that 
user fees are being applied as they were 
designed. In one service of the Agency, 
product label registration, we found that 
regulations were interpreted in a way that 
maximized fee revenue even though this 
was not consistent with the intent of the 
fee’s original design or the amount of 
work entailed in providing the service. 


11.84 Agriculture and Agri-Food 
Canada, through its Review Branch, has 
the capacity to conduct this kind of 
review, but it has not used this capacity to 
review the performance of its user fees. 
The Canadian Food Inspection Agency 
recently formed an audit and review 
directorate and will have the capacity to 
perform such reviews in future years. 


11.85 Have benefits been achieved? 
Review of programs with fees is also 
needed to determine whether charging 
users has achieved the intended benefits. 
Communicating that information to 
stakeholders and parliamentarians, 
especially where non-monetary benefits 
can be identified, can provide some 
assurance that the fees are more than just 
a revenue source. 


Dispute resolution processes are not well 
known 


11.86 We expected that the 
organizations would have established 
dispute resolution processes to deal with 
complaints and disagreements over user 
charges. Such a process is required under 
the government’s policy and is important 
to clients, since it is often their only 
recourse if they are dissatisfied with a 
mandatory (regulatory) service. 


11.87. The Canadian Food Inspection 
Agency has established a process for 
appeals and has published the process. 
However, most of the Agency’s clients we 
interviewed said they did not know that an 
appeal process existed. 


11.88 The Canadian Grain Commission 
and Agriculture and Agri-Food Canada 
have not established formal processes to 
handle complaints about user charges. 
However, the Canadian Grain 
Commission does have a toll-free 
complaints line and an appeal process for 
disputes over grading. Both the 
Commission and the Prairie Farm 
Rehabilitation Administration use 
informal contact and regular meetings 
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with stakeholders to address their 
concerns. 


11.89 A properly designed and 
established appeal and dispute resolution 
process will demonstrate to clients that 
user charges are applied fairly. If clients 
fee] that departments or agencies have not 
given them a fair hearing in the fee-setting 
process, the government’s policy 
explicitly identifies the President of the 
Treasury Board as a “point of contact”. 


What opportunities are there to learn 
from experience? 


11.90 Managers of programs with user 
charges need to learn from experience and 
adjust fees and programs in response to 
changes in conditions and user needs. A 
business in the private sector that does not 
learn from experience jeopardizes its 
future. In the public sector, a program can 
continue to exist even though it does not 
provide services efficiently or effectively. 


11.91 Along with reviewing 
performance, another means of learning 
from experience is talking with 
stakeholders. Consultation on specific user 
charges needs to be a regular, ongoing 
aspect of sharing information about 
program performance and future 
objectives. The Food Production and 
Inspection Branch and then the Canadian 
Food Inspection Agency held annual 
meetings (until this year) with 
stakeholders to discuss fee and program 
changes. While these meetings were a step 
in the right direction, we believe they 
could have been used to better advantage. 
Such meetings are a forum where 
stakeholders can suggest ways to improve 
the programs. They also give the 
organization an opportunity to show 
stakeholders the value they have received 
for their money and to discuss the 
organization’s future goals — and, more 
particularly, user fees. 
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Reports on plans and on performance 
fail to inform adequately 


11.92 Parliamentarians and 
stakeholders have shown a strong interest 
in user charges and they deserve timely 
and complete information. The three 
organizations we audited have 
communicated with users of their services 
about specific fees. However, planning 
documents such as the departmental 
Report on Plans and Priorities and other 
planning documents in the public domain 
contain little information on user charges 
and do little more than carry forward the 
previous year’s experience. There is no 
apparent link between that past experience 
and the way the organization plans to 
achieve its objectives in the future. As 
noted elsewhere, there is no clear 
statement of principles and future plans 
for user charges. This will make it 
difficult for readers of the subsequent 
Performance Report to judge whether the 
actual performance has been satisfactory. 
If this information is considered to be too 
detailed for summary-level planning 
documents, user charge plans could be 
made available to the public and 
parliamentarians. 


Parliamentarians and 
stakeholders have 
shown a strong 
interest in user 
charges and deserve 
timely and complete 
information. 


11.93. The Treasury Board Secretariat 
requests departments and agencies to 
submit an annual summary of planned and 
actual receipts from external user charges 
as well as a listing of their user charge 
initiatives. We looked at the Treasury 
Board Secretariat’s Web site to see what 
information it provides on user charges in 
the organizations we audited. We found 
the information presented to be confusing 
and not very informative (see 

Exhibit 11.4). 


11.94 The report of the Independent 
Review Panel on the Modernization of 
Comptrollership, published by the 
Treasury Board Secretariat in 1997, 
highlighted the need to bring together 
financial and non-financial information 
(both historical and prospective) and to 
establish the links between them as a key 
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to fostering a better understanding of past 
performance — and, more important, to 
support planning for the future. 


11.95 We expected that the 
organizations would provide Parliament 
and stakeholders with a regular report on 
the financial results and performance of 
user charges in relation to their stated 
objectives. As requested by the Treasury 
Board Secretariat, the organizations did 
provide some financial information and a 
list of their user charges. In addition, the 
Canadian Grain Commission and the 


11.96 Organizations need to report 
against the goals established in the 
previous year and provide information that 
will answer the following questions: 


e How do revenues collected from user 
charges compare with forecast revenues? 

e Have the services related to user 
charges been delivered efficiently? 

e To what extent have user charges 
helped the organization meet its 
objectives? 

e Have user charges achieved 
non-monetary benefits? 


e Can readers locate detailed 
information on how specific fees and 
charges were established (costs, unit 
pricing, consultations, legal authorities, 
etc.)? 


Canadian Food Inspection Agency 
provided summary financial information 
in their annual reports. However, none of 
the organizations supplied enough 
information for parliamentarians and 
stakeholders to judge whether user charges 
had been managed properly and the 


services delivered efficiently. (continued on page 11-32) 


Exhibit 11.4 On its Web site, the Treasury Board Secretariat publishes a list of external user charges in 


the federal government along with year-over-year comparisons of associated revenues. 
Based on information supplied by departments and published on a temporary basis until a 
new government-wide accounting system is implemented (Financial Information Strategy, 
planned for the year 2001), this list provides more transparency than existed previously. 
However, the usefulness of the information reported by Agriculture and Agri-Food Canada, 
the Canadian Grain Commission and the Canadian Food Inspection Agency is limited. 


Information on User Charges 
Has Limited Usefulness 


Information on user charges for the 1997-98 fiscal year was not posted until 15 months 
after the end of the year. This meant that until June 1999 the latest information on the Web 
site was for the 1996-97 fiscal year. This does not provide information to users in a timely 
manner. 


There are also problems with the data display that make it very difficult for the reader to 
interpret the information. For example, the list of user charges for Agriculture and 
Agri-Food Canada shows three separate entries for rentals, with no explanation of what is 
included under each. “Rental of Crown-owned Housing and Other Rentals” appears twice, 
along with another entry called simply “rentals”. As well, fee categories are not consistent 
year over year, making it difficult to understand how fee levels have changed. For example, 
the Canadian Food Inspection Agency reports revenue from “Import, Export Fees, Ship 
Inspection, Quarantine, Proceeds from Sales” at $9.2 million in 1994-95 but zero in 
1995-96 and 1996-97. In fact, revenue was received in the latter two years, but was entered 
on the list under a different category. There are also problems with the accuracy of the data. 
This is evident from the amounts shown for the 1997—98 fiscal year, which overstate 
Canadian Grain Commission user charges by approximately $6 million due to the erroneous 
inclusion of parliamentary appropriations. 


This Treasury Board Secretariat document notes, “Additional user charge information can 
be found in the Public Accounts of Canada and in the new Departmental Performance 
Reports.” However, the additional information contained in those sources is also limited 
and difficult to compare, as the categories used by the three agriculture organizations differ. 
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Key Elements and Good Practices for Managing User Charges 


Credible Costing Capability Needed 


What we expected. Organizations need 
to be able to credibly identify their full costs of 
delivering services. They need costing 
information to set amounts for fees and to 
identify and eliminate excessive costs and/or 
inefficiencies in program design and delivery. 
The Treasury Board Secretariat expects 
departments and agencies to follow 
appropriate costing practices. 


Why costing systems are important. 
Without good costing systems, it is difficult to 
know whether user charges are set at 
inappropriate levels. Costing systems also 
allow an organization to compare a program’s 
user charges with its costs and across 
programs, helping to ensure fairness. In 
addition, they allow an organization to 
demonstrate that users are not paying for 
excessive overhead costs. 


Without costing information, it is difficult to 
identify possible efficiencies through 
benchmarking; to evaluate the merits of 
alternative service delivery options; and to 
identify the root causes of favourable or 
unfavourable performance results. 


What we found. None of the 
organizations we audited had produced 


costing information that fully met our 
expectations. For the most part, they use the 
best information they have available but it lacks 
the rigour and sophistication of information 
produced by modern costing systems. For 
example, the Canadian Grain Commission 
recently completed a thorough costing study of 
its services that was a reasonable effort, given 
the cost information it had available. However, 
the costs it measured were captured at a point 
in time only; it does not have any ongoing 
costing systems. As a result, the costs it 
identified were not precise and may not be 
representative. Moreover, the Commission 
cannot compare the operating costs at different 
locations over time. 


Good practice — Net Income 
Stabilization Account. The Net Income 
Stabilization Account (NISA) has developed the 
capability to perform activity-based costing, an 
improvement over traditional cost accounting. It 
identifies the cost drivers for each activity 
performed by an organization. This is a 
sophisticated management tool that, when 
properly designed and implemented, provides 
an organization with reliable and credible 
costing information. 


Failing Grade on Implementing Service Standards 


What we expected. Treasury Board 
policy states, “Fees should be set on the basis 
of clear, and preferably agreed, service 
standards and performance measures unless it 
can be demonstrated that it is not practical or 
reasonable.” As one of the government's 
principles of service quality, clients can expect 
to receive service that “is regularly reviewed 
and measured against published service 
standards” and the results of these reviews are 
to be communicated to clients. Our 1996 
Report Chapter 14 on service quality described 
the government's expectations and the 


progress some organizations had made in 
implementing service standards. 


Why service standards are important. 
Properly established service standards provide 
an objective test of an organization’s ability to 
meet customer service requirements. Regularly 
measuring and reporting actual performance 
against standards helps management identify 
problem areas and assess performance trends 
over time. Where services involve user 
charges, service standards help demonstrate to 
users the value they receive for the fees they 


Pay. 


Officials made a commitment to develop 
an activity-based costing capability for several 
reasons: 


e to better understand their costs and, using 
that information, to improve efficiencies and 
reduce the costs of program delivery; 


e to advise program participants of the impact 
that proposed enhancements to the program 
could have on NISA’s cost structure. This 
information helped participants assess the 
cost/benefit trade-offs involved in building 
more features into the program; and 


e to help NISA set fees for services it performs 
that are outside its core mission. 


NISA has used its activity-based costing 
capability in all of these areas. However, it has 
yet to use it to its full potential in strategic 
planning and improving operational efficiency. 


Conclusion. All of the organizations we 
audited need to significantly strengthen their 
costing capability. Currently only one program 
can produce activity-based costing information. 
The others still have a long developmental 
period ahead before they can meet reasonable 
expectations by users and government for 
costing information. 


What we found. With few exceptions, the 
organizations we audited were not using 
service standards and/or measures of client 
satisfaction as management tools to assess the 
extent to which their services were meeting 
customer expectations. The Canadian Grain 
Commission has developed service standards, 
while the Canadian Food Inspection Agency 
has drafted service standards for most program 
areas. However, neither regularly measures 
performance or reports results. Service 
standards are not an integral part of their 
management culture. 
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Good practice — Net Income 
Stabilization Account. The service standards 
of the Net Income Stabilization Account are 
prominently featured in its annual Producer 
Handbook. Performance is easily measured 
against four of its five standards, and they are 
all readily understandable to producers. Below 
are NISA’s published service standards. 


NISA Service Standards 


e We will process your application and send 
you a Deposit Withdrawal Options Notice 
within 60 days of receiving your properly 
completed application. 


e We will finalize your withdrawal request within 
30 days of receiving request. 


e We will respond to your telephone inquiries 
within 24 hours and to your faxed or written 
inquiries within 15 days of receiving them. 


e We will provide you with an Annual Account 
Statement by February 28 of each year. 


e We will provide you with accurate and 
understandable information concerning your 
account and the NISA program. 


NISA supplements its service standards 
with annual client surveys and focus group 
sessions to try to improve service and validate 
whether existing standards are still appropriate. 
NISA administrators report actual performance 
against standards and the results of client 
surveys at least semi-annually to the NISA 
National Committee (an advisory group of 


Impact Assessments Provide Little Information 


producer clients and federal and provincial 
officials). 


It may be that NISA’s operations lend 
themselves to establishing service standards 
more easily than others we examined. 


Nevertheless, NISA administrators have 
shown that the Treasury Board’s policy 
objective can be achieved. 


Conclusion. Establishing meaningful 
service standards and finding ways to measure 
them is a challenging task, but it can be done. 
Organizations in the Agriculture and Agri-Food 
portfolio need to make more determined efforts 
to develop and use appropriate service 
standards. 


What we expected. Government policies 
for user charges and regulations require that 
before an organization imposes a new user fee, 
it analyze the potential impact on outside 
parties. We expected that impact assessments 
would include examining: 


how individual users were likely to be 
affected; 


whether the fee would be borne by the user 
or passed on to its suppliers or clients; 


whether impacts on different types of users 
would vary significantly; 


whether the fee would put users at a 
significant disadvantage in relation to their 
international competitors; and 


how demand for the service would be 
affected. 


We also expected that assessments of 
cumulative impacts would be completed. 


Why impact assessments are 
important. Assessments ensure that decision 
makers consider impacts before imposing user 
charges, and help alert stakeholders to 
potentially significant changes. 


What we found. The impact assessments 
included in public notices of new regulations 
were usually imprecise and lacked 
quantification. Typical was the Canadian Food 
Inspection Agency's impact assessment of new 
Plant Protection fees, which noted simply, 
“When the fees are prorated to the amount of 
material imported or exported, the increase in 
the cost per item is expected to be small, hence 
the price of consumer goods should not 
significantly increase.” In the case of one new 
fee proposed by Agriculture and Agri-Food 
Canada, the impact assessment did note that 
an expected $4 million in new revenue would 
result, but did not indicate how much more a 
typical borrower would be paying. Some 
assessments provided some information on 
fees in other jurisdictions, but others did not. 


Some Success in Cost Reduction and/or Cost Avoidance 


Good practice - Agriculture and 
Agri-Food Canada. A recent study by 
Agriculture and Agri-Food Canada shed light on 
the cumulative impact that various federal fees 
have had on the agriculture and agri-food 
sector. While the study did have some 
drawbacks in its design, one of the study’s most 
significant achievements was the development 
of analytical tools that could be adapted to 
conduct an impact assessment of any proposed 
fee. The assessment would be used to estimate 
the impact of the proposed fee on user 
profitability and to show how the proposed fee 
would be passed on to consumers, agricultural 
producers or other groups. 


Conclusion. Impact assessments for 
individual fee proposals provided only minimal 
information with which to understand the 
expected impact. This is particularly significant 
because one of the main criticisms levelled by 
industry is that fees have a major effect on 
profitability, and disadvantage them in relation to 
competitors in other jurisdictions. 


What we expected. Before cost recovery 
initiatives are introduced, we expected that 
organizations would, as the Treasury Board 
Secretariat requires, determine whether there 
are better and cheaper ways to deliver the 


service without compromising program 
objectives. 


Why considering cost avoidance and/or 
cost reduction is important. Organizations 
need to ensure that they have considered 


alternatives to new or increased user charges in 
order to alleviate stakeholders’ concerns that 
they have placed more emphasis on revenue 
generation than on redesigning services or 
reducing costs. 
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In order to evaluate alternatives, 
organizations must examine services they 
provide to determine if they are: 


required in order for the organization to fulfil 
its mandate; 


meeting users’ needs; 


being provided in the most efficient manner; 


encouraging appropriate use of federal 
resources by users; 


being charged to the appropriate 
individual/group of individuals; and 


recovering an appropriate proportion of 
direct, indirect and overhead costs. 


What we found. The Canadian Grain 
Commission has re-examined all of the 
services it offers to identify opportunities for 


—————eeeeeeeeee ee 


cost reduction, cost avoidance and/or improved 
efficiency, as well as re-examining existing user 
charges. The other organizations in the 
portfolio have conducted less extensive 
examinations of their services in an attempt to 
address these questions. Some success stories 
have emerged as a result of these efforts. 


Good practice - Canadian Grain 
Commission. In 1998, the Commission, 
working with its stakeholders, achieved a 
significant improvement in operations. In 
response to concerns about the costs of one of 
its services (inward weighing), the Commission 
took advantage of improvements in technology 
and stakeholder input to completely revamp the 
service it had previously provided. A new Grain 
Receival Service was developed that took into 
account both the Commission's statutory 
responsibilities and its clients’ need for reduced 


More Attention Needed to Determining Private Benefit and Public Good 
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costs. The changes resulted in a dramatic 
reduction in unit costs, from $14.75 per rail car 
to $5.90, representing a 60 percent reduction. 


Good practice - Canadian Food 
Inspection Agency. The Canadian Food 
Inspection Agency has reviewed a number of 
its programs. One success story relates to 
livestock grading: as an alternative to increased 
user charges, industry decided to operate its 
own, less costly, grading service. 


Conclusion. The cost of delivering 
programs that meet client needs may often be 
reduced when organizations and stakeholders 
work constructively together. The alternative to 
reducing/avoiding costs — new or increased 
user charges — can spur parties to achieve 
benefits that would not otherwise have been 
realized. 


What we expected. It is government 
policy to implement user charges for services 
that provide identifiable recipients with direct 
benefits beyond those received by the general 
public. Departments are expected to work with 
stakeholders to identify these private 
beneficiaries. They must also determine what 
proportion of the benefit flows to the private 
beneficiaries and what proportion flows to the 
general public. 


Why distinguishing private benefit 
from public good is important. Determining 
the appropriate allocation between private 
benefit and public good is essential to a 
perception of fairness in the fee-setting 
process. Since many government services 
provide both private and public benefits, the 
organizations need to state clearly to 
stakeholders the basis on which they have set 
fees. 


Stakeholders have long been concerned 
about the lack of clearly stated principles and/or 
guidance to help ensure an appropriate and 
consistent approach to allocating private and 
public benefit. The guidance provided by the 


Treasury Board Secretariat has been limited to 
a list of issues to consider. Even with detailed 
guidance, however, the determination of public 
good and private benefit will remain a highly 
judgmental and difficult area for departments to 
manage well. 


What we found. None of the 
organizations we examined have clearly 
communicated to stakeholders a set of 
principles to explain how allocations between 
private benefit and public good are made. 


In response to our 1996 audit of Animal 
and Plant Health, the Canadian Food 
Inspection Agency made a commitment to 
“develop a set of criteria that will guide the 
estimation of the level of private good 
associated with each functional area of 
service”. This work has yet to be completed. 
The Agency has concluded that there are 
private benefits from each of the services it 
provides, and has established a minimum 
private benefit level of 10 percent. However, 
the Agency has not attempted to develop more 
specific guidelines. 


Agriculture and Agri-Food Canada has not 
developed any principles to explain how private 
versus public benefits are identified and valued. 
Such principles would be useful, since there 
are examples of services offered by the 
Department that convey at least some private 
benefits, but where user charges vary from nil 
(Cash Advances program) to partial recovery 
(Net Income Stabilization Account) to full cost 
recovery (Farm Improvement and Marketing 
Cooperatives Loans Act program). 


Most of the Canadian Grain Commission's 
work achieves full cost recovery. However, the 
costs of operating its Grain Research 
Laboratory ($10 million annually) are shared 
equally between users and the government. 
The basis for this 50/50 allocation of 
public/private good, which dates back many 
years, is not readily apparent. 


Conclusion. Stakeholders have 
consistently asked organizations to develop 
clear and understandable guidelines as to how 
private benefit will be distinguished from public 
good. While this is a difficult task, it is one that 
needs to be undertaken. 
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The Consultation Process Has Improved 


What we expected. “Effective and 
meaningful consultation” is the standard 
against which stakeholder consultations are 
measured. We expected that to make 
consultations on user charges effective and 
meaningful, organizations would: 


make reasonable efforts to notify all 
stakeholders of new or amended charges 
before they are introduced; 


give affected parties a reasonable 
opportunity to provide input into the design, 
delivery and pricing of services; 


tell stakeholders why services are delivered 
as they are, how charges are determined, 
and how costs are controlled; and 


provide stakeholders with feedback on what 
was done about concerns they expressed 
and suggestions they made during the 
consultation process. 


Why an effective consultation process 
is important. Consultation is a tool that 
promotes openness and transparency, builds 
general acceptance for proposed changes in 
program delivery and helps to ensure that good 
decisions are made. “User pay, user say” has 
become an important consideration in 
implementing user charges, and the 


consultation process should provide users with 
the opportunity to have their say. 


Consultation ina regulatory environment 
entails particular challenges. Many regulated 
services provide benefits that extend beyond 
those who pay directly for the services; often, 
these regulated services also exist to protect 
the general public. The stakeholder community 
thus includes more than just the “user”. A 
typical example is food inspection, where the 
user may be a processor of foods. The general 
public also benefits from food inspection even 
though it does not pay user charges directly. In 
such cases, it is important that consultations 
reflect the notion of “user pay, user and 
beneficiaries say”. 


What we found. Stakeholders we 
contacted generally reported that consultations 
have improved. The organizations are 
developing stronger relationships with 
stakeholders and have a better understanding 
of the kinds of information stakeholders want. 
With this improved consultation, users have a 
better sense that their concerns are respected 
in program design. 


Good practice - Canadian Food 
Inspection Agency. The Canadian Food 
Inspection Agency developed a set of 
principles that has resulted in a uniform 


Success in Keeping Collection Costs Low 


approach to consultation across its many 
different program lines. The principles also 
clearly laid out a fundamental tenet of the 
Agency — that food safety would not be 
compromised. This fostered a more open 
process and also provided stakeholders with a 
framework to plan and prepare for 
consultations. 


Good practice - Canadian Grain 
Commission. The Canadian Grain 
Commission has been charging user fees since 
1912. Over the years, it has established a 
productive working relationship with its 
stakeholders. It made determined efforts to 
ensure that all players in the grain industry 
were appropriately represented in the 
consultations during the recent comprehensive 
review of its programs. As well, the 
Commission has been careful to balance 
representation among stakeholder groups, who 
at times may have different or even competing 
priorities. 


Conclusion. Over the last several years, 
consultations have improved. Organizations 
could make further improvements by making 
certain that, to the greatest extent practical, 
stakeholders representing all parties, including 
users and beneficiaries, are included in the 
consultative process. 


What we expected. We expected 
organizations to develop efficient mechanisms 
for collecting user charges in order to minimize 
both administrative costs and losses to the 
Crown from uncollectable amounts. 


importance of efficient collection 
mechanisms. Since fee collection as such 
contributes little to the achievement of program 
objectives, its costs need to be minimized. 


What we found. We found several 
examples of well-designed fee collection 
processes and none that indicated any need 
for major improvement. 


Good practice - Farm Improvement 
and Marketing Cooperatives Loans Act. 


The Farm Improvement and Marketing 
Cooperatives Loans Act (FIMCLA) program 
has been designed so that the bulk of program 
administration and loan approval is handled by 
commercial lending institutions. If those 
institutions do not follow proper procedures, the 
loan guarantee by the government may be 
invalidated — a risk to the lenders that helps to 
ensure that they approve only qualified 
borrowers under the FIMCLA program. 


In order to be eligible for a FIMCLA 
guarantee, a borrower must pay the 
government an up-front, one-time registration 
fee of 0.85 percent of the principal value of the 
loan. The lending institution collects this fee 
and remits it to FIMCLA officials. Until they 


receive the fee, the loan is not registered and 
the guarantee does not apply. Consequently, 
fee collection is timely and the government has 
no collectability problems to deal with. 


The government's costs of administering 
the program are thus limited to those of 
interpreting regulations, processing 
applications and managing any defaulted 
loans. Approximately $500 million a year in 
FIMCLA-guaranteed loans and a considerably 
higher amount in previously issued loans are 
managed for approximately $1 million in annual 
administration costs. Only a minuscule portion 
of these administration costs is dedicated to the 
collection of registration (guarantee) fees. 
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Good practice —- Canadian Food 
Inspection Agency. The Canadian Food 
Inspection Agency recently centralized its 
collection activities in one location. 
Previously, collections were managed in 
regional offices throughout the country, with 
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varying levels of efficiency and expertise. 
Now clients across the country have a single 
point of contact for information about billings 
and/or the status of their accounts. With staff in 
one centre who are trained and dedicated 

to this activity, the Agency expects that 


Means of Demonstrating Efficiency Are Limited 
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service will improve and administrative costs 
will be reduced. 


Conclusion. The organizations that we 
audited had developed effective processes and 
procedures to keep the costs of collecting user 
charges low. 


What we expected. Government policy 
requires that organizations explain how costs 
are being controlled in services subject to user 
fees. Hence, we expected that the 
organizations would communicate to users 
persuasive evidence that they are controlling 
costs and operating efficiently. 


Why demonstrating efficiency is 
important. Users should feel confident that 
they are not bearing the costs of an inefficient 
service, especially where services are 
mandatory. 


What we found. We found that with few 
exceptions, organizations did not use 
systematic approaches to demonstrating 
efficiency, for example: 


e benchmarking efficiency against other 
comparable organizations or industry norms; 


e showing how costs and efficiency (e.g. unit 
costs) have changed over time; 


e comparing efficiency of similar activities 
within the organization; and 


e reporting the results of in-depth efficiency or 
resource reviews that identify whether there 
is a potential for significant efficiency gains. 


Not all of these approaches are applicable 
to every situation (for example, tracking 
efficiency over time generally requires program 
costing systems such as activity-based costing; 
benchmarking against other organizations can 
be done only if there are similar organizations 
that will release cost and efficiency information, 
etc.). As well, the extent to which an 
organization pursues more systematic 
measurement of efficiency can depend on the 
extent to which users and managers have 
other means to judge efficiency. For example, if 
users are mostly satisfied from firsthand 
observation that an operation is efficient, then 
more systematic efficiency measurement may 
not be warranted. 


Good practice - Prairie Farm 
Rehabilitation Administration. The Prairie 
Farm Rehabilitation Administration of 
Agriculture and Agri-Food Canada charges 
fees for irrigation services that it provides in 


Existing Appeal Mechanisms Are Weak 


southwest Saskatchewan. In 1996, it proposed 
fee increases for the first time since 1989. To 
justify this, it described how it had tried to avoid 
fee increases in recent years by reducing the 
vehicle fleet, contracting out services and 
downsizing staff. It quantified the total impact of 
these changes, showing that expenditures had 
dropped from an annual average of $1 million 
to $770,000 by 1995-96. 


Good practice - Canadian Food 
Inspection Agency. The Canadian Food 
Inspection Agency (CFIA) has undertaken 
several studies of service efficiency in its meat 
hygiene program — its largest program. These 
studies included an ergonomics study, a 
Regional Overtime System and Industry 
Efficiency Review, and a study titled Service 
Evaluation and Future Needs Assessment: The 
Red Meat Industry. 


Conclusion. Programs should pay 
greater attention to systematically 
demonstrating to users their efforts at cost 
control and operational efficiency. 


What we expected. We expected to find 
that a formal process for reviewing client 
complaints had been established that was 
objective, independent and unbiased. Treasury 
Board policy requires that such a dispute 
resolution process be established. 


Why effective appeal mechanisms are 
important. The opportunity to appeal is often 
the only recourse for a stakeholder to register 
complaints or express dissatisfaction about 
fees for services. Having a means of appeal is 
particularly important in the case of user 
charges for regulated and/or mandatory 


services where government is the sole service 
provider. An appeal mechanism: 


e keeps government accountable for its 
actions; 

e helps to ensure that departments and 
organizations take into account concerns of 
stakeholders; and 

e helps to ensure that departments and 
organizations cost services accurately and 
look for efficiencies in service delivery. 

An appeal mechanism should deal with 
issues such as: 


e the level of service provided for the user 
charge; 


e the user fee process, including consultation, 
accountability and decision making; 


e the general structure of user charges; and 


e how the regulation is being interpreted for 
specific user charges. 


An appeal mechanism exists to deal with 
how the policy of user fees is implemented 
rather than with the government's actual user 
charge policy. A balance may need to be found 
between legitimate complaints of a structural, 
procedural nature and those more specifically 
related to dissatisfaction with the user charge 
policy itself. 
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What we found. Neither Agriculture and 
Agri-Food Canada nor the Canadian Grain 
Commission has developed a formal appeal 
process for resolving user fee/service disputes. 


The Commission does have several 
informal avenues for stakeholders to bring 
complaints/concerns to its attention, including: 


e the Assistant Commissioners, who act as 


The moratorium has 
had the unintended 
effect of freezing 
activity in the 
management of user 
charges. 


Commission, grain companies and 
producers; and 


e a toll-free complaints line. 


The Canadian Food Inspection Agency 


Clients may make formal, verbal and written 


(continued from page 11-26) 


Moratorium on New Fees 


11.97. Ina November 1996 presentation 
to the House of Commons Standing 
Committee on Agriculture and Agri-Food, 
the government made a commitment to 
introduce no new user fees or increases to 
existing fees before the year 2000 (beyond 
those already announced). The intent was 
to reassure Parliament that the proposed 
Canadian Food Inspection Agency would 
achieve efficiencies through cost 
reduction and cost avoidance rather than 
imposing additional fees. 


11.98 Since the announcement of this 
fee moratorium, there have been calls to 
increase its scope and duration. Incomes 
in the agriculture sector have been under 
severe pressure in the form of low 
commodity prices, and the federal and 
provincial governments have developed 
income disaster assistance programs. 
There have also been calls for a 
moratorium on user charges across the 
government. 


11.99 During the course of our 
examination, officials from the 
organizations expressed different views 
about what organizations the moratorium 
applied to and how long it would last. 


Impacts of the moratorium 


11.100 The moratorium has had not only 
the intended effect of freezing fee 
increases but also the unintended effect of 
freezing activity in the management of 
user charges. The pace of implementing 
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ombudsmen and provide a link between the 


has established a process to respond to client 
appeals related to service fees and programs. 


appeals to the Agency. While the Agency has 


communicated the availability of an appeals 
mechanism to its stakeholders, in some cases 
we found that the stakeholders were not aware 
of its existence. 


Conclusion. Appeal mechanisms and 
avenues for dispute resolution for user charges 
need to be formalized and better 
communicated to stakeholders and clients. 


the government’s policy on user charges 
has slowed, and officials point to the 
moratorium as the primary reason. For 
example, consultation by the Canadian 
Food Inspection Agency has declined; and 
there is less emphasis on improving 
management tools such as costing at 
Agriculture and Agri-Food Canada and 
the Canadian Food Inspection Agency. 


11.101 Since Program Review in 1995, 
Agriculture and Agri-Food Canada and 
the Canadian Food Inspection Agency 
have gained experience with user fees and 
increased their knowledge. They have 
made an investment in developing better 
consultations and communicating with 
clients. There is a risk that this capability 
will be set back as the moratorium 
continues. 


11.102 Inthe meantime, programs 
continue to be adjusted to meet the 
changing needs of their users. Examples 
can be seen on a major scale in the 
program redesign at the Canadian Grain 
Commission and the continued 
introduction of the Food Safety 
Enhancement Program (which applies 
Hazard Analysis and Critical Control 
Point principles) in the meat hygiene 
program of the Canadian Food Inspection 
Agency. 


11.103 During the moratorium, however, 
fees cannot reflect these program changes. 
This creates the possibility that some 
clients may pay more than their fair share. 
The divergence of program costs from fee 
revenues can also increase financial risk to 
the organizations, and the risk is 
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especially high for those that rely heavily 
on fee revenues. 


11.104 The moratorium has also raised 
uncertainty among clients and 
stakeholders about the status of previously 
announced plans. There were no clear 
public statements to dispel the uncertainty 
and allow stakeholders to plan 
accordingly. One stakeholder told us, 
“Nobody knows what the future holds for 
user fees — the industry needs time to 
plan.” 


What action is possible? 


11.105 The period of the moratorium is 
an opportunity for organizations to 
evaluate whether user fees are being 
applied properly; to develop measures of 
program efficiency; to develop better 
management tools for user charges, 
including costing systems; and to plan for 
the future. 


11.106 Program managers could also 
collect information on the performance of 
user charges and the impacts of the 
moratorium; and officials could develop 
options for future policy on user charges. 
If the government does not intend to 
continue its implementation of new fees or 
to increase existing fees, one option 1s to 
permit fee adjustments in order to 
eliminate inconsistencies and allow fees to 
reflect changes in services. 


Conclusion and 
Recommendations 


11.107 During the last five years, the 
application of user charges has increased 
significantly in some of the organizations 
we audited. In many cases, officials 
viewed time restrictions and an immediate 
need for revenue as leaving little 
opportunity to design user charges in a 
way that would also provide 
non-monetary benefits. There was also 
little time to look for opportunities to 
avoid or reduce costs. The organizations 
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were met with strong opposition from the 
groups that were obliged to pay the 
increased fees. 


11.108 Each of the three organizations 
has made progress in designing and 
managing user fees to achieve the 
government’s objectives for user charges. 
The structure used for consultations has 
improved significantly over the past 

five years, and in some areas good 
practices have been developed. But there 
are still opportunities to improve their 
management of user charges in a number 
of areas, including costing of services, 
assessing the impact of fees, establishing 
formal appeal processes and integrating 


user charges into strategic planning. Each organization has 


opportunities to 
improve the 
management of user 
charges in a number of 
areas. 


11.109 An opportunity exists to 
strengthen the accountability relationship 
between those who provide the service 
and those who use and pay for the service. 
The organizations can improve services 
and adjust fees based on what they learn 
in applying user charges. This can be 
accomplished by linking better costing 
practices, efficiency of program delivery, 
and improved reporting to users on results. 
The three organizations have not 
monitored and reported their user fees in a 
clear and meaningful manner. 


11.110 Although in recent years they 
have made progress, they are losing 
momentum and user charges are not being 
updated as programs change. This could 
result in some user groups paying either 
more or less than their fair share. There is 
also a risk that the capability to manage 
user fees that has been developed over the 
past few years will be set back. 


11.111 Many officials see the 
moratorium as a sign of a broader 
government reluctance to continue with 
the implementation of user charges. As a 
temporary measure, the moratorium can 
give the three organizations time to 
reassess their current position, develop 
guiding principles, and plan improvements 
for the future of user charges in their 
organizations. 
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Recommendations 


The organizations should take steps to improve their 


Agriculture and Canadian Food Canadian 
Agri-Food Inspection Grain 
Canada Agency Commission 


costing capability in order to improve their management of a a @ 


user charges and to enhance public confidence in the 
administration of user charges. (11.29 to 11.30) 


The organizations should introduce meaningful service 


standards (including measures of client satisfaction) that 


are monitored and reported on a regular basis. (11.31 
11432) 


i © @ S 


The organizations should improve public information on the 
likely impacts of proposed fees, including administrative 8 @ e@ 
and other costs and monetary and non-monetary benefits. 


(11.33 to 11.36) 


Agriculture and Agri-Food Canada should adapt the 


analytical approach used in its cumulative impact study into 


tools for estimating the impacts of individual proposed 
(11.37, Exhibit 11.3) 


The organizations should decide what further study of 


fees. 8 db oles 


cumulative user charge impacts is warranted and plan & ® @ 


accordingly. (Exhibit 11.3) 


Agriculture and Agri-Food Canada should further document 


the cumulative impact study approach and lessons 
learned, and share it with other interested parties. 
(Exhibit 11.3) 


The consultative process for new or amended user 


charges should be broadened to include all significant 


@ N/A N/A 


beneficiaries of the related services, regardless of whether © e © 
or not they are the ones invoiced for the services. (11.41 to 


11.48) 


The organizations should, with the help of central 


agencies, identify ways to increase the responsiveness of 
fee structures to changes in the business environment and 6 a B 
in costs, along with commensurate improvements in tools 


for public and parliamentary scrutiny. (11.53 to 11.59) 


Guiding principles in line with government policy should be 

developed that allow a person to identify situations in which 

user charges will be applied; to understand how private 

and public benefits have been differentiated; to understand e © © 
how the amount of the charge has been determined; and to 


see that the principles have been applied consistently 
equitably across the organization. (11.60 to 11.73) 
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and 
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10. Each organization should ensure that significant user 
charge issues are properly considered in its ongoing ry & 
strategic planning process, and reflected in its strategic 
plan. (11.60 to 11.63) 


N/A 


1: Although the timing may vary depending on the nature and 
significance of the area, the organizations should review 
each of their programs, with and without user charges, to 
identify fee inconsistencies, opportunities to achieve 
benefits beyond revenue generation, opportunities to @ ®@ @ 
simplify the fee structure, and fees at odds with program 
objectives or the organization’s user charge strategy and 
principles. This should identify what changes are needed in 
user charges, and how these changes will be achieved. 
(11.81 to 11.85) 


ee Better means should be developed to demonstrate 
systematically to users that costs are controlled and e B @ 
operations are efficient. (11.81 to 11.82) 


13. The organizations should regularly review (with 
stakeholders) and assess their management of ongoing 
user charge initiatives to ensure that the services being @ @ © 
provided continue to meet the needs of beneficiaries and 
that they are delivered economically and efficiently. 
(11.83 to 11.84) 


14. The organizations should evaluate the extent to which user 
charges have achieved revenue and other benefits, @ @ @ 
and should identify and publicize user charges that achieve 
positive, measurable outcomes for taxpayers. (11.85) 


iis. Appeal and dispute resolution mechanisms should be @ @ r) 
formalized and widely communicated. (11.86 to 11.89) 


16. Where user charges are material, the organizations should 
make significant improvements to the quality of user 
charge information that is made public, so that a @ © 
parliamentarians and the public understand the 
organization’s user charge strategy, principles, plans and 
performance. (11.92 to 11.96) 


17. The scope and duration of the present fee moratorium & r) r 
should be clarified. (11.97 to 11.104) 


Applies 
Applies to a lesser degree (some action already taken, or applies only to certain areas/programs) 
N/A Not applicable 


ee 
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Agriculture and Agri-Food Canada’s 
response: The Department appreciates 
the Auditor General’s recognition of its 
notable successes and innovations over 
the years in administering what is a 
complex and often sensitive file. We are 
always interested in making improvements 
and appreciate the valuable contribution 
of the Auditor General towards this goal. 


Agriculture and Agri-Food Canada is 
developing a management framework 
compatible with portfolio requirements 
that will enable it to better articulate the 
first principles that will guide the future 
application of user fees when the current 
moratorium is lifted. These first principles 
will respect the spirit of the Treasury 
Board policy and the needs of our clients. 
The Department recognizes the value of 
improving costing mechanisms, assessing 
the impact of individual fees and 
establishing appeal processes in the 
management of user charges. We seek 
ways to improve performance in these 
areas when undertaking reviews of 
programs and changes to fees. At the same 
time, the Department is looking for ways 
to improve the requirements and 
management of the regulatory process and 
will seek guidance from the Treasury 
Board Secretariat in this area. 


Balancing good management theory and 
client acceptance of user fees has always 
been a challenge. We are proud of our 
efforts to strike that delicate balance in 
close co-operation with and constant 
feedback from our clients, and in the 
context of other factors which impact our 
working environment. Foremost among 
these other factors are instructions from 
Parliament and the need to balance 
impacts on the sector. 


Canadian Grain Commission’s response: 
The Canadian Grain Commission (CGC) 
appreciates the recognition of its 
successes by the Office of the Auditor 
General. We find that the chapter provides 
useful suggestions for improvement, and 
we intend to act on them. This 


notwithstanding, the following should be 
noted: 


¢ The CGC conducts an annual review 
of all user fees. In recent years, the 
CGC has performed a comprehensive 
review of the costs associated with 
each service. 


¢ The CGC has recently reviewed its 
current position on user fees, restated 
its guiding principles and proposed 
improvements that are in keeping with 
the 1997 Cost Recovery and Charging 
Policy of the federal government. 


e Since 1991, the CGC has taken 
vigorous action, including 
streamlining, reorganizing and 
downsizing, to reduce costs and 
minimize the impacts of fees on the 
users of its services. 


Within the parameters established by 
Parliament, the Commission is committed 
to working with its grain producer and 
industry clients to implement the 
government's cost recovery policy and to 
provide excellent service for the fees it 
collects. 


Canadian Food Inspection Agency’s 
response: The Canadian Food Inspection 
Agency (CFIA) has long contributed to the 
development and implementation of the 
government's cost-recovery policies and 
practices. Experience has shown that user 
charges have many benefits beyond that of 
revenue generation. The CFIA agrees with 
the observations and recommendations 
contained in the audit by the Office of the 
Auditor General. These will assist the 
CFIA to continue to maintain a fair and 
equitable system of user charges for those 
inspection services that provide an 
important economic benefit to the 
Canadian agri-food and fish industries. 


The CFIA is exploring options for 
cost-recovery practices in the year 2000. 
Should the present cost-recovery 
moratorium be lifted, the CFIA will follow 
the Auditor General’s recommendations to 
address the gaps in the current 
cost-recovery policy. Increased revenue 
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generation may well be secondary to more 
important objectives such as the equitable 
and uniform application of service fees 
across all commodity programs. Should 
the moratorium continue, the CFIA will 
focus its activities on possibly simplifying 
fees, reporting on performance, and 
making minor fee modifications to address 
changes to CFIA cost structures, such as 
those resulting from restored collective 
bargaining. 


Agriculture Portfolio — User Charges 


Client consultations will continue to be 
one of the CFIA’s most important 
cost-recovery activities, as they provide 
insight into the economic climate and 
reinforce the need to balance policy 
objectives with broader government-wide 
objectives. An effective, well-designed 
cost-recovery program is one that 
generates revenues in support of program 
objectives, thereby contributing to the 
overall well-being of the Canadian 
agri-food and fish industries. 
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About the Audit 


us 


Objectives 
The objectives of the audit were to determine whether the three organizations we audited: 


e have properly designed and managed user charges so as to achieve the government’s objectives for user 
charges; 


e are learning from experience through the regular review of information and experience gained from 
applying user charges; and 


e monitor and report performance in a way that is clear and meaningful and allows Parliament to judge 
how well user charges have been implemented and managed. 


Scope and Approach 


The audit examined external user charge initiatives in the Agriculture and Agri-Food portfolio, excluding 
Crown corporations and the Canadian Pari-Mutuel Agency Revolving Fund. Our examination included 
Agriculture and Agri-Food Canada, the Canadian Food Inspection Agency and the Canadian Grain 
Commission. Within Agriculture and Agri-Food Canada, we concentrated our work on the Market and 
Industry Services Branch (Farm Improvement Marketing Cooperatives Loans Act), the Policy Branch (Net 
Income Stabilization Account), the Prairie Farm Rehabilitation Administration, and the Research Branch. 


The Office of the Auditor General has examined user charges as part of past audits in the agriculture portfolio. 
In Chapter 9 of our 1996 Report, Animal and Plant Health, we noted that amounts being recovered were 
significantly less than the full costs of delivering the service. There was also a lack of criteria for determining 
private versus public benefit. In Chapter 13 of our 1994 Report, Federal Management of the Food Safety 
System, we noted that costs of food inspection were not being recovered in accordance with the government’s 
policy that was then in place, and that there were inconsistencies in cost recovery practices among and within 
departments involved. 


Using the government’s policy, the Organization for Economic Co-operation and Development and other 
sources of information as a guide, the audit examined how well the various organizations have implemented 
and managed user charges. 


In addition to interviews with staff of the three organizations, we examined and analyzed organizational files, 
data and documentation. We conducted interviews with a number of industry groups as well as individual 
producers who are affected by user charges. We also observed services for which fees are charged by 
inspection staff at various locations, including meat slaughter and processing plants, import and export 
certification, and grain elevators in Vancouver and Thunder Bay. 


The quantitative information in this chapter that was drawn from government and non-government sources or 
departmental databases has been checked for reasonableness but has not been audited. 
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Criteria 
For our first audit objective, we expected that: 


e the three organizations would have a comprehensive user charge strategy with clear objectives, and that 
this would be applied consistently in all areas where user charges may apply across the portfolio; 


e appropriate consideration would be given to options for reducing and/or avoiding costs, and that the 
organizations would be able to demonstrate that activities are delivered efficiently;* 


e _user charges would be implemented in a timely manner, following due process, in accordance with all 
relevant legislative authorities; 


e user charges would be implemented “for services that provide identifiable recipients with direct benefits 
unless overriding policy objectives would be compromised” ;* 


e areasonable analysis of cost drivers and the full cost of providing the service would be performed;* 


e the level of the fee would be established taking into account not only the full cost of providing the service 
but also other factors, such as optimal pricing from an economic standpoint, the proportion of private 
benefit received by the user and the achievement of program objectives; 


e clear and preferably agreed-upon service standards would be established where user charges are applied;* 


e meaningful and effective consultations would be held with stakeholders when fees were established or 
revised;* 


e the formulation of user charges would include information gained from impact assessments that have 
taken into account the cumulative impact of user charges and the impact of fees on competitiveness; 


e the formulation of user charges would include an assessment of the merits of alternative fee structures 
(for example, flat fees versus fees based on the extent of use); and 


e fees would be collected in a timely and diligent manner and that records pertaining to outstanding 
amounts would be properly maintained. 


For our second audit objective, we expected that: 


e aprocess would be in place for the periodic review of user charges, determining whether fees need to be 
adjusted where cost structures have changed, where the mix of public and private benefits have changed, 
or where service levels have been modified. This would include an analysis of results achieved against 
expectations; 


e the information gained from the periodic review and from consultations would be considered for potential 
changes to the services provided, and would be incorporated in the strategy and objectives of the three 
organizations; and 


e program changes would be made in response to the exhibited need for services and feedback from users, 
so that only those services that are needed are provided, and are provided in a manner that best meets the 
needs of the users. 


* Criteria drawn from Treasury Board policy and other Treasury Board documents 
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For our third audit objective, we expected that there would be: 


a long-term plan for user charges clearly communicating to Parliament and other stakeholders the future 
direction of user charges and how these link with achieving the objectives of the three organizations; and 


fair and reliable reporting of financial and performance results achieved by user charges relative to their 
stated objectives provided to Parliament and stakeholders on a regular basis.* 


* Criteria drawn from Treasury Board policy and other Treasury Board documents 


Audit Team 


For information, please contact Neil Maxwell. 


Assistant Auditor General: Don Young 
Principal: Neil Maxwell 
Director: Gordon Stock 


Linda Anglin 
Robert Anderson 
Tina Guthrie 
Martha Lywak 
David Maxwell 
Eimer Quinn 
John Rossetti 
Christian Weber 
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Agriculture and Agri-Food Canada 


A New Crop: Intellectual Property 
in Research 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 
Accountants. The numbered paragraphs in bold face represent recommendations. 
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Agriculture and Agri-Food Canada 


A New Crop: Intellectual Property 
in Research 


Main Points 


12.1 The context of intellectual property related to food production in Canada has changed dramatically in the 
last 10 to 15 years. This is forcing Agriculture and Agri-Food Canada’s Research Branch to seriously rethink its 
way of managing its own intellectual property. It faces the following issues: 


e What should the Research Branch do when it makes a discovery that leads to an invention? Should it 
publish it, or legally protect it and then license it to others? If it does license it, should it license it to 
one organization or many? 


e How should the Research Branch deal with intellectual property of others that it uses in its own 
research? For example, if it develops a new potato by taking advantage of private sector patents, will 
this hinder its ability to get the new product into the hands of producers? Should the Branch buy 
access to private sector patents? Should it barter with its own patents? 


We believe that it is important for the Branch to answer these questions — how best to release its intellectual 
property and how best to acquire the intellectual property of others — because they have implications for the 
future success of agriculture. As Canada’s agriculture sector becomes more and more sophisticated, it depends 
increasingly on exchange of and access to new knowledge coming from the Research Branch and other 
organizations. Answers to these questions also have implications for future government revenues from intellectual 


property. 


12.2 Particularly urgent is the risk to the Branch’s continued ability to provide the agriculture sector with 
innovations. The risk stems from its significant use in research projects of intellectual property owned by others. 
Should it encounter significant obstacles to procuring the rights to use these proprietary technologies, the Branch 
may have to alter, postpone or abandon current research initiatives. 


12.3 The Research Branch is endeavouring to cope with these challenges, but this has become difficult due to 
substantial changes in the Branch and in the agri-food sector. In our opinion, the Branch needs to hasten and 
expand its efforts. A decision framework is needed to guide employees in deciding which research products should 
be legally protected, how to best acquire new intellectual property and how to best release its own protected 
intellectual property. Other tasks facing the Branch are to improve communication with staff; improve information 
on the Branch’s intellectual property; and improve the integration of stakeholders’ views into decision making. An 
overall approach to managing intellectual property is needed. 


Background and other observations 


12.4 Intellectual property is information that is useful and transferable and controlled by someone. It includes 
inventions, some of which are controlled through patents and others that are controlled by secrecy. It also includes 
plant varieties and plant materials. Until recently, there were few ways to protect intellectual property stemming 
from agricultural research, and new crops offered little monetary return to the breeder. New crop types can now be 
protected under the 1990 Plant Breeders’ Rights Act, and some new biotechnology processes can be protected by 
patent. 
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12.5 The Research Branch is the part of the federal agriculture department that does scientific research. It has 
a budget of about $250 million annually and is a key player in agricultural research in Canada. It is a major 
contributor to “Innovating for a Sustainable Future”, one of the Department’s business lines. It first made its mark 
more that 100 years ago with the release of Marquis wheat, which greatly helped develop Prairie agriculture. 
Current revenue from the Branch’s intellectual property licenses is around $3.4 million a year, a minor but 
growing part of its $250 million budget. 


12.6 The challenges the Branch faces in managing intellectual property include the accelerated pace of 
change, especially in the field of biotechnology; uncertainty in the Branch brought on by downsizing and a greater 
focus on research partnerships; the need to consult broadly on intellectual property policy; the high cost of 
protecting and enforcing its intellectual property rights; the need to understand the market for its intellectual 
property; and the need for good management information. 


The Department has accepted our recommendation and has committed its Research Branch to work to 
develop and communicate a general framework for managing the Branch’s intellectual property. 
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Introduction 


12.7 A few years ago it was unlikely 
that a Canadian would become rich by 
coming up with a new plant. There were 
few ways to protect ideas, and new crops 
offered little monetary return to the 
breeder. During the past decade, though, 
the possibilities of making money through 
agricultural research have improved. New 
crop types can now be protected under the 
1990 Plant Breeders’ Rights Act and new 
biotechnological processes can be 
protected by patent. How should the 
research arm of the federal agriculture 
department deal with these developments 
and their implications? 


Role of the Research Branch in 
Agriculture and Agri-Food Canada 


12.8 The Research Branch is the part 
of Agriculture and Agri-Food Canada that 
does scientific research. It is a major 
contributor to “Innovating for a 
Sustainable Future”, one of the 
Department’s business lines. It first gained 
recognition more than 100 years ago with 
the release of Marquis wheat, a pivotal 
crop in the settlement of the Prairies. 
Research Branch innovations continue to 
play a significant role in Canadian 
agriculture (see Exhibit 12.1). In 
developing higher-yielding, hardier, and 
more disease-resistant crops, the Branch is 
recognized as an important factor in the 
continuing success of Canadian farming. 
Departmental economic studies have 
estimated that the Branch’s research has 
brought economic benefits to the agri-food 
sector well in excess of its expenditures on 
research. In 1998-99, the Research 
Branch released roughly 60 new plant 
types and was granted 10 new patents. 


12.9 The Branch has reported that 
while it will collaborate with private 
research firms, it does not want to 
compete head-to-head. It has geared its 
efforts primarily toward science and 
technology that have value to the country 
that the private sector, working alone, 
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cannot provide at a profit. For example, it 
develops plant varieties to fill specific 
niche markets and to deal with special 
challenges faced by Canadian producers. 


12.10 The Research Branch is both a 
user and a producer of intellectual 
property. It obtains intellectual property 
from other research organizations around 
the world and, as a provider, it licenses 
out its own inventions and plant varieties. 
Acquiring and licensing intellectual 
property is one way for the Branch to use 
the products of other research groups in its 
Own activities and then transfer the 
benefits to Canada’s agri-food sector. 


12.11 Depending on the Branch 
management’s goals, and on market 
demand, the eventual revenues from 
licensing its intellectual property may or 
may not become significant. Its 
intellectual property royalties currently 
earn revenue of roughly $3.4 million a 
year, a minor but growing part of the 
Branch’s $250 million budget. 


12.12 More urgent than revenue 
considerations, however, is the need for 
the Branch to maintain its freedom to 


The Research Branch 
is recognized as an 
important factor in the 
continuing success of 
Canadian farming. 


Exhibit 12.1 


The Shepody Potato: A Research Branch Innovation 


Shepody potatoes are good for 
making french fries. Thanks to its 
high yield and relatively early 
maturing, the Shepody potato has 
captured a large part of North 
American french fry production and 
has helped to keep Canadian growers 
competitive. Agriculture and 
Agri-Food Canada estimates that 
about 1.5 billion pounds of Shepody 
potatoes were grown in North 
America last year. The Shepody potato originated at the Department’s Potato 
‘Research Centre in Fredericton, New Brunswick in 1969 and was licensed for 
production in 1980. In recent years the Shepody potato has been the subject of 
biotechnological research, and several genetically modified varieties have now 
been developed. 


We were told that if the Shepody potato had been released with the current Plant 
Breeders’ Rights Act in place, it would now be returning $5 million to 
$10 million a year to the Research Branch. 
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Access to new ideas 
that can be turned into 
readily available crops 
for producers is 
becoming more 
complex and 
expensive for the 
Branch. 


A scientist can patent 
a process used to 
produce a new life 
form, but not the life 
form itself. 


operate on the acquisition and research 
side of its activities. The future success of 
Canadian farming depends on access to 
new knowledge. As agri-food companies 
seek to protect new advances, gaining 
access to new ideas that.can be turned into 
readily available crops for producers is 
becoming more complex and expensive 
for the Branch. 


Federal policy on intellectual property 


12.13. The federal government’s policy 
on managing and recovering intellectual 
property fees is contained in the Treasury 
Board policy on user charges. That policy 
directs that user charges should “earn a 
fair return for the Canadian public for 
access to, or exploitation of, publicly 
owned or controlled resources” and that 
prices should be “based on market value 
for the sale, lease or license”. 


12.14 The ownership of intellectual 
property generated as a result of public 
expenditure is governed by the following 
legislative and policy frameworks: 


e Public Servants Inventions Act and 
Regulations; and 


e Policy on Ownership of Intellectual 
Property in Government Contracts. 


Ownership of intellectual property arising 
out of collaborative work and as a result 
of grants and contributions is governed by 
the terms and conditions of the related 
program or through individual 
agreements. Intellectual property 
generated by the federal government is 
also subject to all laws and regulations 
governing intellectual property in general, 
for example, the Patent Act, the Copyright 
Act, and the Plant Breeders’ Rights Act. 


12.15 In its 1996 federal strategy for 
science and technology, the government 
made a commitment to immediately 
review its policy on intellectual property 
and resolve a number of outstanding 
issues. “Management of Intellectual 
Property in the Federal Government”, on 
page 12-9 of this chapter, provides a brief 


review of the extent to which the 
government has fulfilled this commitment. 


Legal context of intellectual property in 
agriculture 


12.16 There are two main justifications 
for the existence of intellectual property 
rights. First is the philosophical notion 
that someone who comes up with a new 
idea should own that idea, and others 
should recognize the ownership. The 
second justification 1s the incentive to 
engage in research, development and 
commercialization. Protection of 
intellectual property rights brings the 
originator the potential for financial 
return. This, in turn, fosters new research 
and provides for improved products. 


12.17 To obtain a patent, an inventor 
must fully disclose the inner workings of 
the invention. In return, a patent provides 
an inventor with a limited monopoly on 
the new process or product. To be granted 
a patent, an invention must display a 
novel use, or a new or improved way of 
doing something — for example, a better 
mousetrap or a new type of telephone. In 
the agriculture sector, the Canadian Patent 
Office is allowing inventors to patent 
biotechnological processes that can 
produce new characteristics in plants. 
However, to date it has not granted patents 
for inventions that are new higher forms 
of life, such as plants and animals. In 
other words, a Research Branch scientist 
can patent a process used to produce a 
new life form, but not the life form itself. 


12.18 The Plant Breeders’ Rights Act is 
a way of providing “patent-like” 
protection for cultivated plants in Canada. 
However, this protection has some key 
differences from general patent protection. 
For example, the Plant Breeders’ Rights 
Act allows producers to save seed from 
crops they have grown in order to produce 
crops in subsequent years. If a patent 
protected the plant variety, this would not 
be permissible. Various organizations have 
estimated that roughly 70 percent of the 
grain on the Prairies is grown from 
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Management of Intellectual Property in the Federal Government 


What Is Intellectual Property? 


Intellectual property includes inventions; 
computer software; trademarks; literary, artistic, 
musical or visual works; plant varieties and 
other biological materials; and know-how. 
Because it is a potentially valuable asset and 
can yield economic benefits, intellectual 
property warrants effective management, 
including appropriate protection. 


The tools for protection can take different 
forms. For example, inventions may be 
protected by patent and registered industrial 
design; software and literary, artistic and 
musical works may be protected by copyright. 


Intellectual property is generated by the 
federal government in a variety of ways — for 
example, as a result of research conducted by 
federal departments, research contracted with 
outside suppliers, collaborative research, and 
research supported through grants and 
contributions. 


The Science and Technology Strategy 
Commitment on Intellectual Property 


In its 1996 Strategy, Science and 
Technology for the New Century, the federal 
government made the transfer of knowledge 
and technology an explicit objective of its 
activities. The Strategy stated that the existing 
federal policies to encourage the transfer of 
intellectual property from federally supported 
research to the private sector needed to be 
communicated better, applied more evenly and 
assessed regularly for effectiveness. The 
government committed itself to immediately 
review its intellectual property policy in order to 
determine what improvements could be made 
to increase opportunities for commercialization 
and partnerships with the private sector. 


The Government Still Lacks a 
Comprehensive Approach 


We looked at what progress had been 
made in meeting the commitment made in the 
Strategy. While we found that some limited 
initiatives are under way, there is still no 
comprehensive approach to the management of 
all intellectual property generated by the federal 
government. 


Industry Canada is leading an exercise to 
revise the 1991 Treasury Board policy 
governing the ownership of intellectual property 
arising under Crown procurement contracts. 
The objective of the new policy is to increase 
the potential for its commercial exploitation by 
having the ownership rest with the contractor. 
The policy will go into effect six months after its 
approval, following communication of the 
changes and opportunities for departmental 
contracting officers to receive training. 


In late 1998, Industry Canada initiated 
discussions with the Assistant Deputy Ministers 
Committee of Science and Technology about 
the need for a comprehensive approach to the 
management of intellectual property in the 
federal government. Discussions are ongoing 
but a clear action plan is not yet in place. 


In June 1999, the Advisory Council on 
Science and Technology (ACST) published its 
expert panel report, Public Investments in 
University Research: Reaping the Benefits. The 
report makes a number of recommendations to 
government to strengthen the exploitation of 
intellectual property generated by federally 
funded university research. However, the report 
deals with only one facet of intellectual property 
management (federally funded university 
research) and is not linked to a 
government-wide policy. 


In 1997 the Federal Partners in 
Technology Transfer, a forum of representatives 
from 15 departments and agencies, published 
Guiding Principles for the Management of 
Intellectual Property Issues. The principles 
provide guidance on managing intellectual 
property throughout the government. 
Unfortunately, the principles do not have the 
force of policy and their application by 
departments and agencies is voluntary. 


Conclusion 


The federal government develops 
intellectual property in a number of different 
ways. Although it is making progress in the 
management of intellectual property arising out 
of Crown procurement contracts, we have 
concluded that the commitment made in the 
Strategy has not yet been met. A 
comprehensive approach to the management of 
all intellectual property generated by the federal 
government has yet to be developed. 


The challenges identified in the current 
audit of intellectual property management at 
Agriculture and Agri-Food Canada are an 
indication of the growing complexity of these 
issues for departments. A clear statement of the 
government's expectations for how 
departments should manage intellectual 
property would ensure that departments and 
agencies share a common understanding of 
what is expected, and that they have 
reasonable flexibility to establish practices that 
respect their mandates. Many of the building 
blocks are now ready or soon will be. The policy 
on intellectual property arising from Crown 
procurement is being revised. The Federal 
Partners in Technology Transfer have published 
guiding principles for the management of 
intellectual property issues. The Expert Panel 
on the Commercialization of University 
Research has reported to the Prime Minister’s 
Advisory Council on Science and Technology 
and the report’s recommendations are being 
discussed with federal, provincial, university and 
private sector stakeholders. It remains for the 
government to pull the components together 
into a comprehensive policy framework. 
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Many of the Branch’s 
research results have 
broad applications that 
directly benefit 
producers and have 
little licensing 
potential. 


12-10 


producers’ bin seed. As well, the Plant 
Breeders’ Rights Act does not limit 
someone else from using a protected plant 
variety for further research work. Plant 
breeders’ rights protection is considered 
weaker than patent protection. 


The market for Research Branch 
technologies 


12.19 For many companies in the 
private sector, the key to maintaining a 
good return to shareholders is often based 
on the exploitation of intellectual 
property. Those companies invest a great 
deal to develop a technology or product 
that will give a clear market advantage 
over their competition. Companies go to 
great lengths to protect their intellectual 
property to ensure that revenue and 
competitive advantage are not lost. Under 
some circumstances, a company may 
decide that the maximum return can be 
achieved by licensing the protected 
technology or product. 


12.20 A parallel exists in government 
departments that conduct research and 
development leading to new technologies 
and products. However, the market for 
these technologies and the very purpose of 
intellectual property management have to 
be considered in a broader context than is 
the case in the private sector. The focus of 
the Research Branch’s efforts is on 
initiatives for the public good. In some 
circumstances, these efforts can result in 
the development of a technology or 
product that can be transferred to the 
agri-food sector through licensing. The 
Branch makes the transfer in the belief 
that maximum benefit to the taxpayer and 
the agri-food sector will accrue through 
commercial exploitation by the private 
sector. The Branch considers licensing 
revenue to be primarily a tool to ensure 
sound management of the license and the 
associated intellectual property. 


12.21 Many of the Branch’s research 
results have broad applications that 
directly benefit producers and have little 


licensing potential. The main value of 
public research and development in the 
agri-food sector lies in the effect on the 
national economy and not simply in the 
revenue stream from license fees. 


Focus of the audit 


12.22. This work was completed in 
conjunction with the audit of user charges 
in the agriculture portfolio. The objectives 
of our audit were to determine whether the 
Research Branch properly manages the 
acquisition and release of intellectual 
property. We also wanted to determine 
whether the Research Branch uses regular 
strategic review of information and 
experience to improve the management of 
intellectual property. 


12.23. The audit focussed on intellectual 
property that is administered solely by the 
Research Branch. Specifically excluded 
from the scope of our work was the 
Department’s Agri-Food Research and 
Development Matching Investment 
Initiative, under which the disposition of 
intellectual property is determined by 
prior agreement between the Branch and 
private sector research collaborators. 


12.24 More details about how the audit 
was conducted are at the end of the 
chapter in About the Audit. 


Observations 


Challenges Facing the Research 
Branch 


12.25 In managing its intellectual 
property, the Research Branch faces a 
number of challenges — both internal and 
external. In addition to managing the 
acquisition and release of patents and new 
plant varieties, it has to maintain an asset 
management regime that ensures an 
effective portfolio of intellectual property. 
To be effective, the portfolio must: 


e support the goals of the Branch, the 
Department and the government, and be 
able to improve the ongoing 
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competitiveness of the Canadian food and 
agricultural sector through the 
development and transfer of innovative 
technologies; and 


e earn a fair return to the Canadian 
public for access to, and exploitation of, 
these publicly owned resources. 


Limits on the Branch’s freedom to fulfil 
its mandate 


12.26 As noted earlier, perhaps the 
Branch’s most urgent intellectual property 
issue is that its freedom to pursue its 
mandate is being challenged by its use of 
intellectual property developed elsewhere. 
The Branch is using in the order of 100 
technologies that have been patented by 
others. It appears that the use of these 
technologies in its research programs is 
significant, but the full extent is not yet 
known. 


12.27. ‘The Branch’s ability to continue 
providing the agriculture sector with 
innovations will be at risk until it is able 
to secure rights to the technologies that it 
requires. Unless it does so, its research 
products may not be transferable to the 
agri-food sector. The Branch has 
recognized the need to secure the rights to 
use these technologies and is 
endeavouring to develop a co-ordinated 
approach to pursuing licenses to these 
patents. Should it encounter significant 
obstacles to procuring the rights to use 
these proprietary technologies, the Branch 
may have to alter, postpone or abandon 
current research initiatives. 


12.28 Anexample of this situation is 
the case of potatoes that were genetically 
modified at the Branch’s Pacific 
Agri-Food Research Centre to resist 
certain viruses. To modify the potatoes, 
the Research Centre used several patented 
technologies, at least one of which 
belonged to Monsanto Company. When 
Monsanto became aware of the Branch’s 
interest in developing these varieties for 
commercial use, it understandably raised 
the issue of its prior intellectual property 
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rights. Moreover, Monsanto has its own 
interest in developing virus-resistant 
potatoes. To date, this potato has not been 


The Branch’s most 
developed for commercial use. 


urgent intellectual 
property issue is that 
its freedom to pursue 
its mandate is being 
challenged by its use 
of intellectual property 
developed elsewhere. 


12.29 In addition to being limited by 
the intellectual property rights of other 
research organizations, Branch managers 
told us that they are concerned about 
being limited by different approaches to 
intellectual property in other jurisdictions. 
For example, while the Canadian Patent 
Office does not allow the patenting of 
higher life forms, the United States Patent 
Office does. Similarly, there are concerns 
that other jurisdictions may be much more 
flexible in defining what constitutes a new 
invention. For example, a patent 
application was recently filed to grant 
protection for canola with a “yellow seed 
coat” characteristic. Canadian researchers 
consider that the “yellow seed coat” 
characteristic is in the public domain and 
is not patentable because it is commonly 
found in Asian varieties as well as a 
Canadian variety. Branch management 
considers the patent application to be 
relatively narrow and believes that it 
would likely permit the production of 
yellow seeded canola by other means. 
However, some researchers remain 
concerned about the patent’s effect on this 
area of the Branch’s research activity. 


Keeping up with its environment 
The Branch may have 


to alter, postpone or 
abandon current 
research initiatives. 


12.30 According to Industry Canada, 
worldwide agri-food biotechnology 
activity in the year 2000 is estimated at 
US $2.5 billion and is projected to rise to 
US $9 billion by 2005. Scientific 
breakthroughs of the 1970s are resulting 
now in viable commercial products, and 
the pace of change is rapidly increasing. 
In addition, the growing market has 
attracted many more players, and some 
are very aggressive in marketing and 
protecting their intellectual property. 
Intellectual property related to 
biotechnology has been hotly contested, 
with court decisions awarding hundreds 
of millions of dollars to patent holders. 
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A growing market for 
genetically modified 
plants has generated a 
pace of change that 
requires the Branch to 
develop new 
management regimes 
much more quickly. 


Exhibit 12.2 


12.31 So far, the Branch has followed 
an evolutionary, conservative approach to 
managing its intellectual property. It takes 
roughly 12 years to breed a new variety of 
plant, and this amount of time allows for a 
gradual developmental approach to 
management. It took roughly 20 years for 
the Branch to move from essentially 
giving away its varieties in the 1970s to 
using a competitive bidding process today 
as a means to obtain fair market value for 
plant variety licenses. A growing market 
for genetically modified plants has 
generated a pace of change that requires 
the Branch to develop new management 
regimes much more quickly. 


Uncertainty within the Research Branch 


12.32 The Branch has undergone 
significant changes over the past five 
years, not the least of which has been the 
shrinking of its full-time work force from 
more than 3,000 to just over 2,000. At the 
same time, there has been significant 
decentralization of intellectual property 
management and a reorientation to 
collaboration with private sector players. 


Management Uncertainty Results in Different Approaches 


to Licensing Plant Varieties 


future licensee. 


payments. 


Both the Eastern Cereal and Oilseed Research Centre in Ottawa and the Cereal 
| Research Centre in Winnipeg release their respective cereal varieties by 
separately inviting marketing proposals from outside organizations. 


The Ottawa centre checks the proposals to make sure they comply with the 
submission requirements and criteria. These criteria include the past performance 
| of the organization and the proposed marketing plan for the new variety. After 
screening, the organization proposing the biggest initial payment is chosen as the 


The Winnipeg centre evaluates the proposals using a multi-dimensional 
| worksheet that integrates the past performance of the organization, the proposed 
marketing plan for the new variety and the present value of all proposed 


In Ottawa, immediate financial return is the key final decision factor, while in 


_ Winnipeg it is only one of many factors. Which decision process best meets the 
| objectives of the Research Branch? 


The result has been uncertainty about how 
best to manage intellectual property: 


e Managers in the Branch’s research 
centres have devised different approaches 
to commercializing new plant varieties. 
There are differences in their timing of a 
new variety’s release, in their payment 
regimes for new licenses and in the 
criteria they use when evaluating 
proposals to commercialize a variety. 
While this may be partly a reflection of 
different commodities and different local 
markets for different research centres, 
these factors cannot entirely explain such 
a range of approaches (see Exhibit 12.2). 


e During our audit, researchers 
reported that they were not certain how to 
decide whether to publish new scientific 
breakthroughs in journals or pursue legal 
protection for their discoveries. 


e The Research Branch has 
two lawyers on staff for advice on 
intellectual property but, due to the size 
and decentralized nature of research 
operations and the rapid change in this 
area, some researchers and managers are 
uncertain about the legal framework for 
intellectual property. For example, under 
its current arrangements for licensing 
varieties, the Branch grants a licensee 
permission to enforce intellectual property 
rights as circumstances arise. However, 
we were told that the Branch has yet to 
clarify and communicate the 
circumstances in which it will enforce its 
intellectual property rights in its own 
behalf. Given the competitive 
marketplace, the Branch believes its best 
approach is to make decisions to enforce 
its rights on a case-by-case basis. 


Obtaining the views of all Canadians 


12.33, The Department maintains a 
close working relationship with the 
agri-food sector — and particularly, in the 
case of the Research Branch, with seed 
companies and companies engaged in 
agricultural and agri-food-related research. 
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As one Branch employee said, “Everyone 
Knows everyone else.” 


12.34 The form of this relationship is 
complex and varied. Private sector 
researchers and managers have attended 
the same schools as their Research Branch 
counterparts, may have worked at the 
Research Branch in the past, and may 
belong to the same professional 
organizations. In addition, a number of 
companies ensure that they have staff 
tasked with keeping track of developments 
taking place in the Research Branch 
laboratories spread across the country. The 
relationship between the Research Branch 
and its client group is very close. 


12.35 Given this close relationship with 
the agri-food sector, the Branch faces the 
challenge of ensuring that its approach to 
intellectual property management reflects 
the views of all interests, including those 
of producers and the general public. The 
Branch has a number of consulting 
mechanisms that it uses to determine its 
research priorities and co-ordinate the 
transfer of research and technology with 
other organizations. These mechanisms 
include the Canadian Agri-Food Research 
Council, the Research Branch Advisory 
Committee and advisory committees at 
the Branch’s research centres. However, 
we did not identify a systematic approach 
to consulting these or other groups about 
the Branch’s management of intellectual 
property. During the audit, researchers 
told us that in the absence of such a 
mechanism they feel that they have 
become the sole guardians of the public 
good. 


Difficult resource decisions to be made 


12.36 The Research Branch’s budget 
was cut by approximately 20 percent 
between 1995 and 1998 and it lost 
approximately 1,100 full-time equivalent 
employees. While intellectual property 
decisions once were managed centrally 
from Ottawa, they have largely been 
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devolved from the Research Branch 
headquarters to the research centres. 


12.37 The resources needed to protect : : - 
intellectual property are considerable and We did not identify - 
compete directly with other resource systematic approach 
requirements of the research centres. We to consulting 


were told that it can take up to $100,000 
to support a patent submission. We were 
also told that the process can take one 
year and a substantial amount of a 
researcher’s time away from ongoing 
research. This means that decision makers 
are faced with the difficult question of 
whether the future benefits of patenting 
will exceed the future costs. 
Biotechnology researchers told us that 
they could each probably come up with 
one or two patentable ideas every year if 
they had adequate support. 


12.38 The research centre directors 
with whom we spoke could not point to 
any formal guidance from Ottawa on how 
best to make decisions on the use of 
intellectual property resources. 


interested groups 
about the Branch’s 
management of 
intellectual property. 


Understanding the market for 
intellectual property 


12.39 Acquiring, selling and protecting 
intellectual property requires substantial 
organizational resources and knowledge 
about the market for intellectual property. 
Management needs to understand the 
market to set realistic expectations for 
new intellectual property and to evaluate 
the performance of its existing portfolio. 
The Branch conducts market studies to 
support research planning. Also, it 
provided us with a study in which it had 
evaluated the market for a new 
technology, but we were unable to identify 
any other systematic assessments of the 
market for Branch intellectual property or 
an assessment of the impact of its 
licensing regime. Researchers reported 
that they often have a sense of what the 
market for their research products might 
be. However, they also contended that in 
their role as research scientists they should 
not be the ones responsible for knowledge 
of the market. 


The research centre 
directors with whom 
we spoke could not 
point to any formal 
guidance from Ottawa 
on how best to make 
decisions on the use 
of intellectual property 
resources. 
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Research centre 
directors, managers 
and researchers need 
a structure for making 
decisions about 
intellectual property. 


12.40 To help support the licensing of 
intellectual property, the Branch has 
created commercialization officers at the 
research centre level. These people assist 
in patent filing, and are also tasked with 
helping to negotiate matching investment 
agreements and managing the release of 
new plant varieties. It is unlikely that they 
would have the time to spend on a 
strategic assessment of the intellectual 
property market. 


Lack of information for managing 


12.41 To manage an asset, whether a 
building or intellectual property, an 
organization needs to know how much of 
the asset it has, its location, and its value 
in terms of both its ability to generate 
revenue and its broader public benefits. 
The Research Branch is unable to fully 
answer these fundamental questions. For 
example, it could readily tell us how many 
plant varieties were earning revenue but 
could not easily tell us how many released 
varieties it currently has, and how many 
are generating no revenue. 


12.42 Similarly, the Branch needs to 
define management’s performance 
expectations for its portfolio of 
intellectual property. It also needs to 
develop a reporting regime that 
adequately recognizes the dual objectives 
of providing Canadian research products 
that help to keep the agriculture sector 
competitive and earning a fair return on 
publicly owned resources. Performance 
expectations could, for example, be set for 
the financial return to the portfolio, or the 
extent to which producers adopt new 
varieties. 


12.43 In recent years, the Branch has 
made some progress in that it is able to 
report how many plant varieties it has 
released in a year. It can also report on 
significant earnings. The need to improve 
information systems in all 
research-oriented departments was 
recognized by the government in its 1998 


update of the federal Science and 
Technology Strategy. 


The Way Forward 


12.44 The challenges the Branch is 
facing are mostly due to changes in the 
Branch and in its environment. In some 
cases, the changes have a direct bearing 
on the commitment the Branch has made 
to innovate for a sustainable future for 
agriculture. These changes can hamper the 
Branch’s ability to provide research 
inventions to the agriculture sector and 
can make it possible for other 
organizations to patent the Branch’s 
research output and potentially restrict its 
use. To address these challenges, Branch 
management needs to hasten and expand 
its efforts in some key areas. 


Framework to help guide decisions 


12.45 Research centre directors, 
managers and researchers need a structure 
for making decisions about intellectual 
property. A decision-making framework 
would foster a consistent approach and 
help to ensure that the goals of the 
Branch, the Department and the 
government are met. This is particularly 
important in a decentralized 
“knowledge-based” organization like the 
Branch, with research centres spread 
across the country. 


12.46 A framework would support a 
number of key decisions that the Branch 
needs to make, for example: 


e whether or not to protect or publish 
new scientific breakthroughs and how best 
to arrange the licensing of a technology or 
a plant variety. Among other 
considerations are the need to protect a 
future stream of revenue, and to guarantee 
future access for Canadians to the 
Branch’s results; 


e how to develop a portfolio of 
intellectual property that can be bartered 
in exchange for intellectual property held 
by the private sector; 
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e how aggressive the Branch should be 
in protecting its intellectual property 
rights; and 


e whether to license a new technology 
from the patent holder or to develop the 
Branch’s own solution. 


The Research Branch is not alone in the 
need to deal with these issues. Some of 

them were faced in the first patenting of 
biotechnological inventions in the 1980s 
(see Exhibit 12.3). 


12.47 — Starting points for developing a 
decision framework are the Branch’s own 
1993 technology management manual and 
the government’s current review of 
intellectual property. The technology 
management manual needs to be updated 
to bring it into line with the changes that 
have taken place in the Branch. For 
example, the manual needs to reflect the 
role of the commercialization officers, the 
loss of the Branch’s Industry Relations 
Office and the closer relationship that the 
Branch has with industry in areas such as 
the Matching Investment Initiative. It also 
needs to consider the more aggressive 
environment in which it must operate. The 
challenge will be to develop an effective 
decision framework that ensures some 
level of consistency while supporting the 
development of a more entrepreneurial 
culture, able to deal with the uncertainties 
of the commercial world. Exhibit 12.4 
shows the major elements of what we 
would expect to see in a decision 
framework. 


Communication with Branch employees 
to foster an entrepreneurial culture 


12.48 Tied to the development of a 
framework for decision making 1s the 
need to clearly communicate the 
framework and foster an entrepreneurial 
culture among Branch employees. This 
communication could take many forms, 
including a training program. No matter 
what vehicle it chooses, however, the 
Branch needs to ensure that its employees 
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understand its approach to intellectual 
property and that they understand and are 
secure in the role they play. We were told 
that information sessions on legal issues 
surrounding intellectual property are 
already being held. These could probably 
be developed and expanded to include 
briefings on the decision framework and 
to emphasize an entrepreneurial approach 
to intellectual property rather than the 
traditional bureaucratic approach. 


The Branch needs to 
ensure that its 
employees understand 
its approach to 
intellectual property. 


12.49 Along with communication, the 
Branch could put in place mechanisms 
that would help foster an entrepreneurial 
approach. These might include, for 
example, changes in the way the 
Department rewards and recognizes the 
development of intellectual property. 


Enhanced management information and 
development of performance measures 


12.50 The Branch has made some 
moves to improve its information systems 
but it needs to do more. It is currently 
developing a database of all protected 
technologies used in its laboratories, and 
assessing whether to pursue licenses from 
their owners or replace them with the 
Branch’s own technologies. At the time of 
our audit, the Branch had collected the 
data but had not yet checked them for 
accuracy or compiled them in a usable 


Exhibit 12.3 


Dealing With Objectives for Intellectual Property: 
The Stanford University Experience 


In 1980, after dealing with resistance from its own researchers, Stanford 
University obtained one of the first biotechnology patents ever granted. Then, 
according to Niels Reimers, former director of Stanford’s Office of Technology 
Licensing, it faced additional problems. How was the university going to “develop 
a licensing program consistent with the public service ideals of the university, 
encourage the application of genetic engineering technology for public use and 
benefit, minimize the potential for biohazardous development, and finally, provide 
a source of income for educational and research purposes?” 


To meet these objectives, Stanford granted non-exclusive licenses that allowed 
access to all potential users of the patent. It also devised a rate scheme based on 
how the invention would be used in an eventual product, and included 
disincentives for use outside the United States. In addition, the license included 
guidelines on safe use of the invention. 
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form. Still needed also is a centralized 
system to keep track of the Branch’s plant 
varieties and the performance of all the 
Branch’s intellectual property. 


12.51 The Branch needs to establish 
management expectations against which 
to measure the performance of its 
portfolio of intellectual property. As we 
have noted, expectations could be stated 
in the form of the financial return to the 
portfolio or the extent to which producers 
adopt new varieties. Another step, one that 
private sector firms undertake, could be to 
measure and then seek to reduce the 
amount of valuable research in the Branch 
that is unpublished or unprotected. An 


‘intellectual property audit” and the help 
of market experts could facilitate this. 


Strengthened consultative tools 


12.52 The purpose of consultations 
between government departments and 
stakeholders is to allow people a voice in 
decisions that affect their lives, to improve 
the quality of those decisions and to 
provide transparency in the 
decision-making process. Given its central 
role in ensuring the success of Canadian 
agriculture, the Branch’s approach to 
intellectual property affects a number of 
Canadians. To help ensure that it develops 
an effective decision-making framework 


Exhibit 12.4 


Elements of an Effective Decision Framework 


| The objective of a decision framework is to help ensure that all members meet the goals of the organization. The intent is to overcome 
the restrictions of distance and time while dealing with new circumstances. Senior management cannot afford to make all decisions, 

| nor can it abdicate decisions to individual members of the organization and reasonably expect that management’s vision will be 

achieved. 


An effective decision framework sets boundaries and outlines priorities for decision making. The degree to which the framework is 
prescriptive depends on the goals of the organization, its structure, and the nature of its activities. Given the dynamic nature of the 
environment surrounding the Research Branch’s intellectual property, a less prescriptive approach would likely be appropriate. 


The elements of a framework to support decision making on intellectual property in the Research Branch would apply to both the 
acquisition and the release of intellectual property. Some suggested elements are listed below: 


e The goals of decision making on intellectual property would be clearly stated and explained. The goals concerning the public good 
would be specified and any financial expectations stated. These would set out the Branch’s response to Treasury Board’s user 
charge policy requirements. 


e The goals would be translated into strategic objectives and the financial and other resources needed (e.g. legal, marketing expertise) 
would also be outlined. These would support priorities among decisions and would help to communicate organizational constraints 
on intellectual property decisions. 


e Measures to determine if the organization is achieving its strategic objectives would be identified, stated and put in place, and 
would relate achievements to the resources expended. 


e A description of decision-making roles wauld specify who is responsible for what decisions and the scope of decisions that can be 
made. The scope of decision-making responsibility would be defined by the magnitude of the possible outcome. 


e The key intellectual property decisions required by the Branch would be identified. 


Guidance would be provided on the timing of intellectual property decisions. Key factors would be the stage of an intellectual 
property's development and the expected value of that property in terms of both the public good and financial return. 


e The requirements for information that might reasonably be factored into decisions would be outlined. 


e Specific examples of “good” decisions and practices would be included, or communicated as they develop. This would help ensure 
that best practices in decision making are spread through the organization. 


e The framework would be actively supported by Branch senior management and organizational responsibility for implementing and 
maintaining the framework would be stated. 


e A timetable io regularly review the decision framework would be included. 


Source: Developed by the Office of the Auditor General 
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for intellectual property, the Branch needs 
a way to incorporate a broad range of 
concerns. 


12.53. Agriculture and Agri-Food 
Canada has developed sophisticated ways 
to ensure that it interacts with a broad 
spectrum of stakeholders when it is 
making strategic decisions. Similarly, the 
Research Branch has mechanisms to help 
it obtain advice on what research it should 
undertake and on the transfer of 
technology. The Branch now needs to take 
advantage of these and perhaps other 
consultative tools to help it develop a 
strategy for its intellectual property. 


Conclusion and 
Recommendation 


12.54 This audit had two objectives. 
We wanted, first, to assess the quality of 
intellectual property management in the 
Research Branch, and second, to 
determine whether it is improving its 
management regime. Based on our audit 
work, it is apparent that the Branch needs 
to deal with a number of significant issues 
before we can say that it is adequately 
managing intellectual property. 


12.55 Particularly urgent is the risk to 
the Branch’s continued ability to provide 
the agriculture sector with innovations, a 
risk that stems from its use in its research 
projects of intellectual property owned by 
others. Should it encounter significant 
obstacles to procuring the rights to use 
these proprietary technologies, the Branch 
may have to alter, postpone or abandon 
current research initiatives. 


12.56 There are clear indications that 
Branch management recognizes the 
challenges it faces and has begun efforts 
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to deal with issues surrounding its 
freedom to operate. These efforts need to 
be hastened and expanded. The Branch 
needs to build a framework for better 
decision making, improve its information 
systems, broaden its consultations, and 
clarify roles for employees in a way that 
will foster a more entrepreneurial culture. 


12.57 Agriculture and Agri-Food 
Canada should ensure that the Research 
Branch develops and implements an 
overall approach to managing its 
intellectual property. The approach 
should be appropriately communicated 
to staff and support the development of 
a more entrepreneurial culture. 
Implementation should include the 
development of a decision framework 
for Branch staff, a statement of 
management expectations against which 
to measure performance, and the use of 
appropriate information systems. 


Department’s response: The Department 
is pleased that the Office of the Auditor 
General recognizes that the management 
of intellectual property is a rapidly 
changing field and presents a complex 
challenge to research organizations in 
both the public and private sectors. 
Agriculture and Agri-Food Canada has 
been working to address these challenges 
as they have developed over the past 
nine years. 


The Department welcomes 
recommendations on improving the 
management of any intellectual property 
issue and appreciates the useful 
suggestions for improvements made by the 
Auditor General. The Research Branch 
will work to develop a general framework 


for managing its intellectual property, and 


to communicate this approach to staff. 


Based on our audit 
work, it is apparent 
that the Branch needs 
to deal with a number 
of significant issues 
before we can say that 
it is adequately 
managing intellectual 
property. 
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%y About the Audit 


Objectives 
The objectives of our audit were to determine whether Agriculture and Agri-Food Canada’s Research Branch: 
e has properly managed both the acquisition and the release of intellectual property; and 


e is using regular review of information and experience to improve the management of intellectual 
property. 


Scope and Approach 


The audit focussed on intellectual property that is administered solely by the Research Branch. Specifically 
excluded from the scope of our work was the Department’s Agri-Food Research and Development Matching 
Investment Initiative. Under that initiative, the disposal of intellectual property is determined by prior 
agreement between the Branch and private sector research partners. 


We based our criteria on the Treasury Board user fee policy, private sector practices and available literature 
on intellectual property management. 


We visited three research centres and evaluated and compared their respective regimes for managing 
intellectual property. Specific items of intellectual property were examined to determine how property has 
been licensed to the private sector. This work was supported by interviews with senior Branch management, 
research centre directors and staff. 


Criteria 
We expected that: 


e the Research Branch would have a comprehensive intellectual property strategy with clear objectives, 
and that this would be applied consistently in all areas where intellectual property existed in the Research 
Branch; 


e the Research Branch would properly communicate the strategy to Branch employees so that managers 
and staff would understand it as well as their roles and responsibilities in managing intellectual property; 


e meaningful and effective consultations would be held with stakeholders in relation to intellectual 
property management; 


e performance expectations for intellectual property management would be set and appropriate information 
would be collected to assess whether the stated expectations were being achieved; 


e aprocess would be in place for the periodic review of the management of intellectual property and this 
would include an analysis of results achieved against expectations; and 


e the information gained from periodic review and from consultations would be considered for potential 
changes to intellectual property management and would be incorporated in the strategy and objectives of 
the Research Branch. 
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Assistant Auditor General: Don Young 
Principal: Neil Maxwell 
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Marilyn Taylor 


For information, please contact Neil Maxwell. 
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Chapter 13 


National Defence 


Hazardous Materials: Managing 
Risks to Employees and the 
Environment 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 
Accountants. The numbered paragraphs in bold face represent recommendations. 
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Main Points 


13.1 Our audit at 10 Canadian Forces bases found widespread, frequent and recurring instances in which 
National Defence did not meet the legal and policy requirements that govern hazardous materials. Such instances 
may put employees and the environment at increased risk. 


13.2 The Department has made efforts to improve its management of hazardous materials. While there are few 
clear measures of the impact of these efforts, we did find that injury rates for hazardous material injuries that 
require at least one day off work declined by more than half from 1993 to 1997. 


1335 The 10 bases that we audited do not periodically monitor their air or liquid effluent emissions for all 
hazardous material contaminants on which limits are set by federal laws or guidelines. As a federal agency, the 
Department is not legally bound by provincial or municipal laws and bases do not monitor emissions and effluents 
against all of those standards. 


13.4 There is a lack of information on the overall state of compliance with legal and policy requirements. The 
existing management systems to ensure compliance at the base and unit levels do not promote continual 
improvement. Objectives are not set, plans do not exist, audit and inspection programs are not consistent and 
performance information is lacking. These are major factors in the instances of non-compliance by the 
Department with legal and policy requirements. 


Background and other observations 


1S: National Defence uses more than 6,000 hazardous products, such as gasoline, ammunition and battery 
acid. The Department estimates that at least one quarter of its 80,700 full-time employees come into frequent 
contact with hazardous chemical products, and that every worker may have some contact with hazardous 
materials. 


13.6 It is the Department’s policy that its hazardous materials management activities must meet or exceed the 
letter and spirit of applicable federal acts, regulations, policies and guidelines and, where appropriate, be 
compatible with provincial acts and municipal and international standards. The Department has not defined the 
meaning of “where appropriate”. 


13.7 Our audit focussed on how the Department is implementing its policy and programs related to hazardous 
materials. We looked at policies, procedures and results at National Defence headquarters, and conducted field 
work at 10 bases. We also reviewed how the Department is implementing relevant sections of its sustainable 
development strategy. 


13.8 Leadership is needed to set targets, to identify and monitor performance measures, and to follow up to 
ensure that programs are implemented. Commanders and managers have opportunities to demonstrate leadership 
when they revise the Department’s sustainable development strategy and implement environmental management 


systems. 
The Department responded positively to all our recommendations and proposes to take action that should 


address the problems our audit identified. In particular, the Department told us it is committed to rectifying 
situations of non-compliance with the requirements of the Workplace Hazardous Materials Information 
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System; reviewing its effluent monitoring procedures and plans for air emissions; identifying provincial and 
municipal regulations and policies applicable at each base and wing; and establishing core requirements for 
audit and inspections to ensure consistency. 


The Department did not fully accept our conclusions about the level of non-compliance, contending that its 
audits have identified relatively few instances each year. 
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Introduction 


139 Industrialized societies use many 
products that are regulated because they 
can harm people and the environment — 
for example, by exploding, catching fire, 
corroding, poisoning or irradiating. These 
hazardous materials include acids used in 
industry to etch metal, radioactive 
components in smoke detectors, most 
paints and paint thinners, fuels and oils, 
and bleaches and solvents for cleaning. 
Apart from their widespread use in the 
workplace, most Canadians would 
probably find a broad range of hazardous 
materials in their garage, in their garden 
shed, and under their kitchen sink. 


Canada has extensive legal 
requirements governing hazardous 
materials 


13.10 Industrial societies will continue 
to use many hazardous materials until 
environmentally friendly substitutes 
become available at acceptable cost. 
Organizations and societies must therefore 
manage and minimize the risks of 
continued use. In Canada, an extensive 
and growing body of laws has established 
minimum standards that govern the use of 
hazardous materials. 


13.11 Federal laws govern labelling, 
safety instructions, training, protective 
devices, emergency response procedures, 
transportation documents, packaging, 
releases into the environment, storage, 
explosives and radioactive materials. 
Provincial laws parallel many of the 
federal laws but also govern outdoor air 
quality, transportation of hazardous 
materials within a province and disposal 
of hazardous wastes. Municipal by-laws 
regulate discharges of liquid effluent to 
sanitary and storm sewer systems. All of 
these laws establish a number of 
mechanisms to ensure the protection of 
workers and the environment. 


13.12 The Workplace Hazardous 
Materials Information System (WHMIS) 
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is a national information system designed 
to protect Canadian workers. It is a 
systematic approach to identifying the 
hazards of a material, making safety data 
on the material available in the workplace 
and ensuring that workers receive 
appropriate training. The system groups 
materials into classifications based on 
their hazardous characteristics, and uses a 
set of symbols to identify the main 
classes. WHMIS consists of three 
components: 


e A series of labels provided by either 
the supplier or employer indicate, at a 
minimum, the product name, safety 
information and a reference to a material 
safety data sheet. 


An extensive and 
growing body of laws 
has established 
minimum standards 
governing the use of 
hazardous materials. 


e Material safety data sheets (MSDS) 
provide detailed information about 
products, such as ingredients; fire, 
explosion, reactivity and toxicological 
data; protection and emergency response 
measures; and contact names and 
telephone numbers. Suppliers are 
responsible for providing the sheets with 
their products, and employers are 
responsible for providing them to workers. 
The sheets must be updated every 
three years or whenever new information 
about the product becomes available. 


e A training program for workers 
teaches them how to interpret information 
on labels and material safety data sheets, 
particularly those on the specific products 
they use in their workplace. 


13.13. For WHMIS to be effective, 
workers must have access to accurate and 
complete inventory lists of the hazardous 
materials they use. The lists allow workers 
to ensure that they have the correct MSDS 
on hand and that they are trained for the 
specific products they use. The lists also 
tell emergency response personnel the 
types and magnitude of hazards they may 
encounter in responding to a spill, fire or 
injury, so that they can protect themselves 
and deploy appropriate equipment. 
Inventory lists also assist in monitoring 
usage of products to ensure that minimum 
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quantities are acquired and products are 
not duplicated. 


13.14 In addition to classification and 
symbols under WHMIS, the 
Transportation of Dangérous Goods Act 
establishes a second system of 
classification and symbols for 
transportation and storage. The 
classifications in the two systems are 
similar though not identical, and the 
symbols are entirely different. The 
symbols of both systems are often used 
together on a product. 


13.15 Proper storage is essential to 
protect workers and ensure that hazardous 
materials do not escape into the 
environment. Ideally, their storage should 
be apart from other materials, have 
limited access and meet recognized 
standards. Depending on the material, 
storage facilities may be designed to 
withstand fires or explosions or to contain 
spilled material. They may be heated and 
ventilated, and equipped with particular 
types of emergency systems such as 
vapour detectors, sprinklers and 
explosion-proof lighting. Limits on the 
size and storage capacity of facilities 
depend on the nature of the material. 
Because many hazardous materials react 
when they come into contact with each 
other and produce an explosion, fire or 
toxic gas, there are various tables and 
compatibility charts to guide workers in 
identifying incompatible products so they 
can be stored separately. 


13.16 Some types of activities and 
facilities require licences or permits — for 
example, air emissions, storage of 
radioactive materials and ammunition, and 
facilities that generate hazardous waste. A 
licence or permit may provide standards 
against which to check quantities and 
activities, or may require tracking of 
activity over time. 


13.17. For many hazardous materials, 
there are legal limits on the quantities to 
which workers can be exposed or that can 


be released into the environment. A 
worker’s exposure to hazardous materials 
can be measured by requiring periodic 
medical checks, conducting air quality or 
swipe tests of the workplace, or providing 
radiation workers with dosimeters. (Our 
audit of the Department’s measuring of 
workers’ exposure was limited to 
observing some respiratory protection 
practices at bases.) 


13.18 Where their exposure levels 
could exceed allowable limits, and other 
means to eliminate the hazard are 
unavailable or ineffective, workers must 
be provided with personal protective 
equipment, properly fitted, and be trained 
in its use. We did not audit the availability 
or appropriateness of protective 
equipment at National Defence facilities. 


13.19 Releases into the environment 
through air emissions, sewage or 
stormwater also can be measured 
periodically to establish that legal limits 
on concentrations of hazardous materials 
are not exceeded. 


13.20 Emergency response equipment 
and trained emergency response personnel 
are to be available to limit injuries and 
prevent the further escape of material into 
the environment. We did not audit the 
availability of emergency response 
capabilities at National Defence. 


National Defence uses many hazardous 
materials 


13.21. National Defence defines 
hazardous materials as “any material that, 
if handled improperly, can endanger 
human health and well-being or the 
environment or equipment. Some 
examples... are poisons, corrosive agents, 
flammable substances, ammunition and 
explosives.” This chapter, unless 
otherwise specified, uses the term 
“hazardous materials” to include 
ammunition and radioactive materials as 
well as hazardous chemicals and 
polychlorinated biphenyls. 


13.22 The Department uses more than 
6,000 hazardous products. Exhibit 13.1 
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illustrates typical activities at a Canadian 
Forces base, and examples of the types of 
hazardous materials that are used. 


13.23. According to estimates by the 
Department, at least one quarter of its 
80,700 full-time employees come into 
frequent contact with hazardous chemical 
products in the workplace. In addition to 
those are 293 military and civilian 
ammunition technicians who have 
frequent contact with ammunition; and 
1,132 employees who wear dosimeters 
because they work with the more 
radioactive materials in the Department’s 
inventory. Many contractors also come 
into contact with hazardous materials on 
departmental property, and so may the 


Every employee comes into 


some contact with 
hazardous materials during 
the course of the day (see 
paragraphs 13.23-13.24). 


29,400 reservists in the course of their 
part-time duties. 


13.24 The Department estimates that 
every worker, like many in light industrial 
facilities and offices, comes into some 
contact with hazardous materials during 
the course of the day. Soldiers may work 
with ammunition or cleaning solvents for 
weapons. Even office workers may use 
cleaning supplies or toner for photocopiers 
and printers. 


Departmental policy on compliance 
with provincial and municipal laws is 
not clear 


13.25 National Defence is exempt from 
all or part of many federal laws governing 
hazardous materials. In general, the 
exemptions exist to allow the Department 
to take actions during military operations 
that might not be appropriate during 
peacetime or by civilians. As a federal 
agency, the Department is not subject to 
most provincial and municipal laws. 


13.26 It is the Department’s policy that 
its hazardous materials management 
activities “must meet or exceed the letter 
and spirit of all applicable federal acts, 
regulations, policy and guidelines and, 
where appropriate, be compatible with 
provincial acts, regulations and guidelines 
and municipal and international 
standards.” National Defence has not 
defined what it means by “where 
appropriate”, and as a result it is not clear 
in what circumstances the Department’s 
actions will be compatible with 
provincial, municipal or international laws 
and standards. For example, the 
Department conforms to provincial 
regulations on hazardous waste 
generation, and recently made a 
commitment to be compatible with the 
applicable standards in provincial and 
municipal laws governing sewage 
treatment plant and stormwater 
discharges. However, the Department has 
made no commitments to meet provincial 
laws governing air quality and permits for 
potentially contaminated air emissions, 
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nor to meet the mainly municipal laws 
governing sanitary sewer discharges to 
municipal systems. 


Focus of the audit 


13.27. This audit focussed on how 
National Defence is implementing its 
policy and related programs on hazardous 
materials. Specifically, it was intended to 
determine whether the Department: 


e minimizes the risks to its employees’ 
health and safety and to the environment; 


e has systems in place that are 
effective in ensuring ongoing compliance 
with applicable laws, regulations and 
policies; and 


e has established appropriate 
sustainable development targets for 
hazardous materials and can measure its 
performance in meeting them. 


13.28 Further information on our audit 
scope, criteria and approach can be found 
at the end of the chapter in the section 
About the Audit. 


Observations and 
Recommendations 


Risks to Employees and the 
Environment 


Inadequate implementation of 
protective mechanisms increases risk 


13.29 As we have noted, there are 
protective mechanisms that Canada 
requires in the workplace where hazardous 
materials are used (paragraphs 13.12 

to 13.20). When these protective 
mechanisms are not in place, workers and 
the environment are placed at increased 
risk, of two kinds: 


e accidents, leading to injuries and 
contamination; and 


e ongoing exposures, leading to 
chronic illness and contamination. 
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13.30 In fact, National Defence’s rate 
of injuries related to hazardous materials 
and requiring at least one day off work fell 
by more than half from 1993 to 1997. The 
injury rate among military personnel has 
been consistently low in comparison with 
workers performing similar tasks in 
industry. The rate among civilian 
employees has fallen by two thirds 

since 1993 and is now lower than levels in 
comparable industries (see Exhibit 13.2). 
However, both the military and civilian 
rates increased in 1998, although the rates 
remain below | per 1,000 employees. 

At the time of our audit, comparable 
information from industry was not 
available for 1998. 


From 1993 to 1997, the 
rate of hazardous 
material injuries that 
require at least one 
day off work declined 
in National Defence by 
more than half. 


13.31 Information on contamination is 
more problematic. While the Department 
requires the reporting of hazardous 
material spills, until this year (1999) some 
discretion in reporting was allowed; 
moreover, the information was entered 
into databases that are not compatible. 
Spills considered “significant incidents” 
were reported to National Defence 
headquarters; 58 such spills were reported 
in 1998, ranging from half a litre to 
17,500 litres. The Department recently 
issued a spill reporting directive and 
developed a spill reporting computer 
program, which may address the concerns 
about inconsistent reporting requirements 
and incompatible databases. 


13.32 Environmental baseline studies 
and subsequent assessments conducted at 
all Canadian Forces bases since the 
mid-1980s have identified significant 
contamination caused by hazardous 
material use and accidents. National 
Defence has developed a database to track 
its contaminated sites, many of which it 
believes were contaminated by leaking 
underground storage tanks or by 
technology and management practices 
used in the past. Remediation and 
monitoring costs are borne by National 
Defence headquarters for sites 
contaminated before 1993, and by bases 
for their sites contaminated since then. 
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Efforts have been 
made to improve 
storage facilities, 

the availability of 
emergency response 
equipment, and spill 
response training. 


Exhibit 13.2 


Hazardous Material Injuries - 
National Defence and Industries 
Performing Similar Tasks 


Source: Association of Workers’ 
Compensation Boards of Canada; 
Statistics Canada; National Defence 
Directorate of General Safety 


The contaminated sites database lists 

530 sites on the 10 bases we audited, and 
headquarters expects to spend 

$17.3 million on them over the next five 
years. While it is likely that most of the 
sites were contaminated before 1993, the 
database does not identify when the 
contamination occurred. We did not audit 
the management and remediation of sites 
where soil and groundwater contamination 
have been identified. 


13.33 As well as clean-up costs, 
financial liabilities from claims, fines or 
prosecutions can result from failure to 
implement legally required protective 
mechanisms. To date, the Department has 
pleaded guilty twice for violations of the 
Fisheries Act, and has been ordered to 
pay $42,021. 


Compliance in some areas has not 
improved since 1993 


13.34 Inrecent years, National Defence 
has made a significant effort to improve 


Injuries per 1000 employees 
PPS) 


some aspects of hazardous materials 
management. Since 1993, headquarters 
has made available approximately 

$9 million through the Corporate and 
Incremental Environmental Programs to 
improve storage facilities, purchase 
emergency response equipment, and 
provide spill and emergency response 
training. Bases have also acquired 
equipment with their own budgets. Efforts 
have been made to identify and 
disseminate good practices. A database of 
best practices, most of which address 
hazardous materials, has been created for 
the Land Maintenance System. National 
Defence headquarters is developing a plan 
to implement the practices, where 
operationally feasible and cost-effective. 


13.35 Given these efforts, we attempted 
to assemble an overall picture of 
compliance with hazardous materials laws 
and policies at the 10 bases we audited. 
We examined the reports of all external 
and departmental audits and inspections 
conducted from 1993 to 1998 in order to 
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— Defence Civilians 


Utility 


--—_ _ _ 


\_/ Transportation 


Construction 


= pent pr hepato te 
ost. Other Federal Departments pee 
Ss b { 
— — ome “SSE 2) ees eee 
aa a a ce re ma nets Se Oe ees 
0 National Defence Military 
T T T mt 
1993 1994 1995 1996 1997 


13-12 


Report of the Auditor General of Canada — September 1999 


National Defence — Hazardous Materials: Managing Risks to 
Employees and the Environment 


identify instances of non-compliance. We 
also conducted audits of compliance at 
three bases: Cold Lake, Gagetown and 
Halifax. Finally, we examined all the 
reports on Safety Program Development 
and Evaluation Technique (SPDET) 


assessments conducted from 1993 to 1998. 


These are self-evaluations conducted by 
units and bases; they provide a rating out 
of 100 points for each of 16 components 
of an organization’s safety program. The 
technique includes a section specifically 
on hazardous materials safety, but several 
other sections also contain relevant 
elements. 


13.36 We found that many different 
types of departmental inspections have 
been conducted at the 10 bases, generally 
reflecting the different responsibility areas 
of the base safety and environment 
officers. Inspection frequency has varied 
widely, though it has generally increased 
in the last two years. Inspections of 
National Defence facilities by other 
federal departments responsible for 
enforcing hazardous materials laws are 
also uneven in frequency and coverage. 
We found that inspection methods were 
rarely similar across a base, or at the same 
unit for two or more years. 


13.37 Because of the variety of 
inspection methods and the gaps in 
coverage, the inspection data cannot be 
aggregated to provide performance 
information on trends or the overall state 
of compliance, at individual bases over 
time or across bases. The data indicate, 
however, that from 1993 to 1998 there 
were more than: 


e 800 instances at the 10 bases when 
applicable federal legal requirements 
governing hazardous materials were not 
met; 


e 450 violations of the federal Fire 
Code; and 


e 200 instances of failures to meet 
departmental or federal policy 
requirements. 


13.38 The audits and inspections also 
identified approximately 120 additional 
instances where best management 
practices were not followed. 


13.39 Complying with legal and policy 
requirements ensures that the minimum 
standards or risk levels established by 
regulatory authorities or the Department 
are met. The degree to which a failure to 
comply with legal and policy 
requirements increases the risk inherent in 
the use of hazardous materials is very 
dependent on the circumstances. A 
relatively minor oversight, especially if 
combined with others, can lead to a 
serious accident. Examples can be found 
in the Department’s widely distributed 
safety publications. While individual 
instances listed in the Department’s audit 
and inspection reports may have been 
corrected, we found that certain problems 
seem to recur over time and across bases 
(see Exhibit 13.3). 


13.40 We observed the common 
compliance failures listed in Exhibit 13.3, 
as well as several others, during our audits 
of Canadian Forces bases at Halifax, 
Gagetown and Cold Lake. There was a 
noticeable difference among the 

three bases in the general levels of 
management and compliance. We 
consider Halifax to be in the best state of 
compliance, with Gagetown next; Cold 
Lake faces significant compliance and risk 
issues. A general summary of what we 
found at the three bases 1s provided in 
Exhibit 13.4. 


13.41 The frequency of Safety Program 
Development and Evaluation Technique 
assessments is also uneven. None had 
been conducted at two of the bases in our 
sample, and the number conducted each 
year at the remaining eight facilities 
varied widely. There is no obvious trend 
in the results (see Exhibit 13.5), but the 
average ratings achieved in the SPDETs 
appear quite high; the Department 
considers that any rating of 75 percent or 
higher is acceptable. We question whether 
a rating of 75 percent for handling 
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Inspection data 
showed more than 


1,400 instances when 


legal and policy 
requirements 


governing hazardous 


materials were not 
met. 
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hazardous materials can be considered 13.42 A department-wide internal audit 
acceptable: 90 percent of the points in of hazardous materials management was 

SPDET assessments are for meeting legal conducted in 1991-93. The audit included 
requirements designed to protect the site visits at four Canadian Forces Supply 


health and safety of workers. Moreover, as Depots and their support bases, two of 
self-assessments, the SPDETs may contain which were also in our audit sample. The 


a bias toward positive results. We note in internal audit observations were similar to 
particular the contrast between the our own in a number of compliance areas, 
situation we observed at Cold Lake during __ particularly inventory information and the 
our audit and the high average rating of use of WHMIS. We concluded that in the 
94 percent achieved in SPDETs conducted _last six years, compliance in these areas 
there in 1998. has shown little or no improvement. 


A isonet minor oe can lead to a serious Check Fuel Levels Safely, Don’t “Check Out”! 
accident (see paragraph 13.39). 
soldier, let's call him Charlie, was on guard duty at a 
bivouac site when the heater ran out of fuel during the 
ns evening. He and several other soldiers went to get some fuel. 
A Bad Mix It was dark and there were many jerry cans, both full and 
empty, in the petrol, 
Canadian Forces serviceman approached the floor- oil and lubricants 
washing task with gusto. To obtain the best results (POL) point. One of 
possible, he decided to concoct a stronger cleaning his companions, who 
solution, so he mixed the contents of two very similar and we will call Ralph, 
poorly marked opaque plastic containers — vinegar and picked up a full can 
bleach — in the bucket. next to one marked 
WRONG! Our zealous serviceman was overcome by diesel and pro- 
the gasses given off and required hospitalization as a ceeded to fill 
result. Luckily, it doesn’t appear that he will suffer a the heater. 
permanent disability or any long term harmful health He wasn't 
effects. it could have been fatal. When combined, able to see if the 
vinegar and bleach produce a heater tank was ful ; —_ 
potentially lethal gas. so he put the can down 
Make sure that to check. At the same time, Charlie decided to use his lighter 
cleaning fluids are (there was no flashlight) to check the fuel level. He reasoned 
kept in well-marked that diesel fuel fumes do not readily ignite when subjected to 
and labelled containers, open flame. With a "flick of his Bic,” he transformed himself 
read the labels into a human torch. He had ignited the fumes, which were 
carefully, and don't coming from the tank. It turns out that Ralph had inadvertently 
mix incompatible filled the heater tank with gasoline instead of diesel. Charlie's 
material upper body was engulfed in flames, which caused second 
degree burns to his face and neck. While his recovery was a 
long and painful process, Charlie's injuries will not result ina 
permanent disability. 


Adhering to the following safety rules would have prevented 
this near tragedy: 


DND Safety Digest - Edition 1 1998 
‘ ef (_} Store POL products properly and ensure they are 


clearly marked and segregated. 


(_) Ensure that flashlights (and filters if tactically required) 
are available if night operations are planned. 


(_} Never use any type of open flame around fuel 
containers or when refueling vehicles or equipment. 


DND Safety Digest - Edition 2 1998 
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The Department has not succeeded in 
fully implementing the Workplace 
Hazardous Materials Information 
System 


13.43. Compliance with all three 
elements of WHMIS is weak. In a sample 
of 229 hazardous products, the 1993 
internal audit found that 38 percent were 
not labelled. Departmental audits and 
inspections since 1993 have noted 
inadequacies in WHMIS labelling every 
year at Cold Lake, and frequently at five 
other bases. At all three bases we 
examined in more detail, we saw 
unlabelled drums at several locations and 
hazardous liquids dispensed from drums 
into smaller containers that often were 
unlabelled. 


13.44 Even weaker is the 
implementation of the second WHMIS 
component, the availability of up-to-date 


material safety data sheets for all 
hazardous materials used. Departmental 
inspections that included WHMIS found 
deficient MSDS at every base in every 
year for which data were available, except 
the base at Wainwright in 1998. There 
were 76 instances of deficient MSDS 
reported at the nine other bases in 1998. 
Our three in-depth base audits found 
hazardous products base-wide whose data 
sheets were unavailable or out-of-date. 


Lack of container 
labelling is a problem. 


13.45 We distributed a questionnaire to 
collect data on a sample of hazardous 
chemical products recorded on inventory 
lists at the 10 bases in our audit. Sample 
selection focussed on locally procured 
items, though some of the responses 
indicated that the selected item had been 
obtained through the national system. 
Further details about the sample selection 
process and response rates can be found in 
the section About the Audit. Of the 


Exhibit 13.3 


Ten Most Common Hazardous Material Non-Compliance Findings 1993 to 1998 


| Finding 


1. Inadequate or inappropriate storage locations 


| 2. Deficient material safety data sheets (MSDS) 


3. Inadequate and inappropriate signs on storage facilities 


4. Inadequate emergency response procedures 


5. Training or training records missing 


6. Inappropriate management procedures 


7. No inspection process 


8. No secondary containment 


9. Inventory lists lacking 


10. Workplace Hazardous Materials Information System 


(WHMIS) labels missing 
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| 
Risk or Hazard 


Risk of spill or fire, which could endanger personnel and the 
environment 


Information on handling procedures, first aid and hazard 
information unavailable to personnel using the material 


No warning to employees and emergency personnel of 
presence of hazardous materials 


Risk to personnel or the environment due to proper procedures 
not being followed 


Danger to personnel who do not know the proper safety 
procedures 


Risk of injury to personnel handling the material improperly 


Risk to personnel or the environment due to undetected 
problems in hazardous material storage 


Possibility of spills entering the environment or exposing 
personnel to additional risk 


Risk that necessary training requirements are not determined 
and danger to firefighting personnel who will not be aware of 
the presence of hazardous materials 

Danger to personnel who are unaware that they are using 
hazardous materials 
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607 responses in our sample, only 
28 percent had an up-to-date MSDS. 
We noted three types of deficiency: 


hazardous materials in the Department has 
made it a laborious process to keep MSDS 
up-to-date. This is because different 


products may be in each of the widely 
scattered inventories of hazardous 
materials at departmental facilities, and 
the age of the MSDS for each product 
must be tracked and a new one obtained 
periodically. Information technology 
increasingly offers a solution to this 
problem, provided the available 
information meets Canadian legal 
requirements. Many MSDS are now 
available directly on the Internet. More 
can be found on commercially available 
CD-ROMs, and making these accessible 
through internal computer networks can 
expand their distribution. 


e 30 percent were missing MSDS; 


e 16 percent had MSDS that were over 
three years old and therefore out-of-date; 
and 


¢ 26 percent used internally produced 
hazardous materials guide sheets. Since 
February 1994, Labour Canada (now part 
of Human Resources Development 
Canada) has not considered the guide 
sheets to be legally acceptable substitutes 
for MSDS. 


13.46 These results are very similar to 
the findings of the 1993 internal audit that 
examined the MSDS for a sample of 329 
items. That audit found that 39 percent of 
the items were missing the MSDS and 

12 percent had MSDS that were 
out-of-date. 


13.48 The Department has been 
developing its own hazardous materials 
information system to include electronic 
versions of MSDS. The Hazardous 
Materials Information Query, or Hi-Q, has 
been under development since 1996 but 
responsible authorities at several bases 


13.47 The devolution of responsibility 
for the purchase and management of 


Exhibit 13.4 


Summary of Compliance Findings at Three Canadian Forces Bases 


CFB Halifax, 29-30 September 1998 


Hazardous materials are generally stored, handled and used in compliance with regulatory and National Defence requirements in 
buildings and on ships; hazardous materials and wastes located on the jetties are not always handled in accordance with these 
requirements. Staff interviewed during the audit were generally knowledgeable and aware of the handling requirements for hazardous 
materials and wastes. 


CTC Gagetown, 17-18 November 1998 


Hazardous materials management is very decentralized. Each unit is largely responsible for hazardous materials procurement, storage, 
handling, and disposal. A number of employees responsible for hazardous materials management within their units have not received 

| Workplace Hazardous Materials Information System (WHMIS) training or spill response training. While some units appear to have good 
practices in place, improper handling practices, such as unlabelled containers and ungrounded drums used for dispensing flammable 
solvents, were observed at a number of locations throughout the base. This inconsistency indicates a lack of co-ordination and control 
over hazardous materials management. A number of locations where significant quantities of hazardous materials are stored are not 
secured to prevent spills or leaks of dangerous goods from entering the natural environment. 


4 Wing Cold Lake, 14-15 October 1998 


There is a high frequency of significant legal compliance issues associated with hazardous materials management. Some of these could 
entail a high level of risk. There is evidence of incremental contamination as a result of poor handling practices at the Hazardous Waste 
Facility, and many locations where significant quantities of hazardous materials are stored are not secured to prevent spills or leaks from 
entering the natural environment. While some squadrons appear to have good practices in place, the inconsistency in the application of 
these practices indicates a lack of co-ordination and control over hazardous materials management. The volumes of hazardous waste 
generated at this base appear high relative to the use of materials required for base activities. 
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indicated that they have had problems 
getting the system to work. However, 
version 3.0 (distributed in early 1999) may 
address some of these concerns. Initially, 
it was available only as a stand-alone 
system on CD-ROM, but Hi-Q has since 
been integrated with the Materiel 
Information Management System that is 
used mainly at base supply sections and 
headquarters. 


13.49 Making Hi-Q more widely 
accessible will not alone resolve the 
problem of MSDS availability. When we 
compared MSDS on the Hi-Q CD-ROM 
with the 607 questionnaire responses on 
our product sample, we found up-to-date 
data sheets for only 41 percent of the 
responses (although some did not identify 
the product clearly enough to be sure it 
was the product on the disk). With Hi-Q in 
place, more MSDS on our sample items 
should now be available, but that will 
depend on access to Hi-Q information at 
the work site. Responsible authorities in 
the workplace will have to supplement 
Hi-Q with other sources of MSDS 
information, as they do now. We found, 
for example, that Hi-Q did not include 

74 percent of the up-to-date MSDS we 
obtained on our sample. 


Rating (%) 
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13.50 The Department should ensure 
that responsible authorities at National 
Defence workplaces meet the 
requirement to make available 
hazardous materials safety and 
environmental information, particularly 
material safety data sheets. 


Department’s response: The Department 
is committed to rectifying situations where 
it is out of compliance with the 
requirements of the Workplace Hazardous 
Materials Information System. 
Accordingly, the Department will review 
and revise its standard operating 
procedures at all levels so that Material 
Safety Data Sheets (MSDS) accompany 
new products introduced into the 
Canadian Forces Supply System and 
introduced at Bases and Wings. The 
Department will establish a system to 
track the effective dates of MSDS and will 
review and revise its standard operating 
procedures to ensure that all data sheets 
are kept current. The revised standard 
operating procedures will define the 
responsibilities and the accountability of 
all personnel involved in these processes 
and will also address the issue of local 
procurement. 


100 
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Exhibit 13.5 


Safety Ratings — Self-Assessment 


by Eight Bases 


Source: Safety Program Development and Evaluation Technique reports from Borden, Cold 
Lake, Edmonton, Esquimalt, Gagetown, Halifax, Montreal and Trenton, 1993-1998 
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Forty-five percent of 
personnel in our 
sample lacked training 
in interpreting 
information in the 
Workplace Hazardous 
Materials Information 
System. 


Hazardous materials 
inventory systems 
need to include 
standard information. 


There is a lack of training in hazardous 
materials 


13.51. The third component of WHMIS 
is training. We examined the training 
records of a sample of 454 personnel 
selected at random from the 

6,238 employees that the 10 bases 
identified as having access to selected 
hazardous materials inventories 
(information on our sample selection and 
response rates can be found in the section 
About the Audit). Since these individuals 
have access to hazardous materials, we 
expected that they would all have received 
training in interpreting WHMIS 
information. Both the Canada Labour 
Code for civilian employees and 
departmental policy for military personnel 
require that records of training be kept. 


13.52 The departmental records 
available indicate that 205 of the 
employees in our sample, or 45 percent, 
did not have any WHMIS training. This 
compares with the 1993 internal audit, 
which found that 42 percent of a sample 
of 1,783 personnel had not received 
training. 


13.53. Of the 249 workers in our sample 
who had received training, 28 had no 
record of it on their files. We found the 
information in other sources, such as 
safety officers’ databases. 


13.54 We also found poor record 
keeping on specific training in packaging, 
storage and handling, and transportation 
of hazardous materials. These courses are 
required only for personnel who work 
extensively with hazardous materials, such 
as supply technicians. A certificate is 
issued to each trained worker and it must 
be renewed every one to three years, 
depending on the qualification. Expired 
certificates must be kept on file for at least 
two years. Of the 32 workers in our 
sample who required some type of 
training in the packaging, storage and 
handling or transportation of dangerous 
goods, only 21 had a current 
recertification. Seven had been employed 


for several years before they received their 
first training. Few certificates, current or 
expired, were kept on file. 


13.55 Appropriate and up-to-date 
records of training in hazardous materials 
are important because, first, they are 
required by law and by the Department’s 
policy. Second, they also ensure that 
training can be delivered efficiently: 
personnel could be overtrained or 
undertrained if no records exist to show 
when their training is to be updated. This 
is particularly important in an 
organization like National Defence, with a 
significant proportion of personnel 
moving frequently from one job to 
another. 


Inventory lists are inaccurate and 
incomplete 


13.56 Availability of accurate, complete 
and informative inventory lists was 
another area where frequent problems 
were identified by departmental audits and 
inspections, the 1993 internal audit, and 
our three base audits. At Halifax and 
Gagetown we found that inventory lists 
were not available at all locations where 
hazardous materials were stored. Where 
they were available, they were often 
out-of-date and did not accurately reflect 
the current inventory of hazardous 
materials — a problem also observed at 
Cold Lake. Across the 10 bases in our 
sample, departmental audits in 1998 noted 
21 instances of missing, incomplete or 
outdated inventory lists of hazardous 
materials or hazardous waste. 


13.57 We requested copies of all 
inventory lists of hazardous materials at 
the 10 bases we examined. In spite of the 
fact that the Department’s hazardous 
materials safety manual provides clear 
guidance and examples showing how to 
prepare inventory lists, there was little 
uniformity in the documents we received. 
They ranged from handwritten lists of 
chemical products, with incomplete 
identification and no indication of 
quantities, to computerized base-wide 
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inventories with detailed product 
information, container sizes, quantities 
and storage limits, locations, points of 
contact and availability of MSDS. 


13.58 Again, information systems 
technology, including commercially 
available database applications, would 
seem to offer a solution. This has been 
recognized at National Defence 
headquarters and at three of the bases in 
our sample; however, each has developed 
its own system. This has resulted in 
duplication of effort and wide variations 
in the type of information available. 


13.59 The Department should ensure 
that hazardous materials inventory 
systems are implemented and that they 
contain standard information. 


Department’s response: The Department 
will define and issue minimum information 
requirements to be included in Base- and 
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Wing-level hazardous materials inventory 
systems. 


13.60 Responses we obtained on our 
hazardous products sample also revealed 
that inventory lists were not up-to-date. 

A total of 17 percent of items on which we 
received a response could not be found, 
had never been used by the unit, or were 
no longer used. In a number of cases, units 
reported having disposed of the product 
once our request had alerted them to the 
fact that they had it in inventory. 


13.61 We noted that many of the locally 
procured products were purchased more 
than one year ago. In several cases, we 
found more of the product in inventory 
than had been purchased; in other cases, 
the full amount purchased was still in 
inventory more than one year later. Both 
circumstances suggest that material had 
been purchased unnecessarily or in 
excessive quantities. 
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Inventory lists show little uniformity. They range from handwritten lists, 


like the one here from Halifax, to computerized systems with detailed 
information like the inventory from Montreal (see paragraph 13.57). 
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Storage practices are deficient 


13.62 At many bases, storage facilities 
do not meet legal and policy requirements. 
Departmental and external audits and 
inspections in 1998 found a total of 

162 instances of non-compliance with 
storage requirements, with at least some 
instances at all 10 bases. At the three 
bases we audited, we observed the 
following non-compliance: 


e many flammable storage cabinets at 
Cold Lake and Gagetown are metal office 
lockers, rather than regulation 
fire-resistant cabinets with containment 
for spills and proper ventilation; 


e storage and handling areas at Cold 
Lake and Gagetown are not protected 
against leaks or spills, whether by 
secondary containment or paving and 
sealing to prevent seepage into the 
ground; 


e incompatible products are stored 
together and compatibility information is 
missing at Cold Lake, Gagetown and 
Halifax; and 


e storage areas are not properly 
marked with signs or placards at Cold 
Lake, Gagetown and Halifax. 


Respiratory protection programs are 
not fully implemented 


13.63 The use of hazardous materials at 
National Defence facilities generates 
several types of potentially contaminated 
air emissions. While all bases we audited 
have assessed work areas for airborne 
contaminants, we observed these improper 
respiratory protection practices: 


e paint being mixed in unventilated 
rooms at Cold Lake and Halifax; 


* personnel using inappropriate 
respiratory equipment at the hazardous 
waste area at Halifax; and 


e incomplete testing of the fit of 
respirators at the Bedford Ammunition 


Depot, Borden schools, Esquimalt, 
Gagetown and Montreal. 


Monitoring of Continuing 
Emissions 


Outdoor air emissions are not 
adequately monitored 


13.64 Emissions of hazardous materials 
can contaminate the air outdoors. Sources 
of such emissions at Canadian Forces 
bases include: 


e the burning of fuels in heating 
systems and vehicles that can emit large 
volumes of pollutants, including those that 
contribute to climate change; 


e paint spray booths operating with 
forced-air ventilation that can result in the 
release of toxic chemicals, particularly if 
filter systems are not properly maintained; 
and 


e poor storage practices, including 
open containers of hazardous material. 


13.65 Federal guidelines govern 
opacity, sulphur dioxide and particulate 
levels in emissions from boilers. Of the 
10 bases in our sample, Edmonton and 
Wainwright do not have central heating 
plants. Of the remainder, only Cold Lake, 
Halifax and Gagetown monitor for some 
of the pollutants whose emissions are 
limited by the federal guidelines. None 
monitor for all three of the contaminants 
on which limits are set. 


13.66 The provinces set limits on the 
concentration of pollutants in the air, and 
provincial laws require that approvals be 
obtained for all sources likely to affect 
ambient (surrounding) air quality. The 
approvals list on a case-by-case basis the 
monitoring or other action required to 
ensure that the concentration limits are not 
exceeded. Because federal facilities are 
not subject to provincial laws, National 
Defence does not apply for provincial 
permits. Nonetheless, Cold Lake, Trenton, 
Halifax, Bagotville and Montreal monitor 
for at least one pollutant whose 
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concentration is limited by provincial law. Sewage treatment program meets 

No base indicated that it monitored paint standards for biological contamination 
booth emissions, though Borden did at low cost 

indicate that contractors regularly verify 13.67. Hazardous materials can 

the efficiency of its paint booths. contaminate water leaving National 


(See paragraph 13.62) 


Waste oil pan being emptied into a floor drain in a garage at Gagetown. 


Dirty and deteriorating dry filters inside a 
paint booth at Gagetown. 


Unlabelled pails of waste stored outside of the Drums of oil, solvent and WD10 stored on grass with no secondary 
heating plant at Halifax. containment, alongside a garage at Cold Lake. 
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Changes in sewage 
treatment plant 
operations have met 
water discharge 
standards at a low 
cost. 


Defence facilities through storm sewers 
and sanitary sewers. Possible sources 
include: 


e run-off from vehicle or aircraft 
maintenance and storagé areas; 


e discharges resulting from ship 
maintenance; 


e run-off and discharges from 
construction sites and fabrication shops; 
and 


e spills. 


13.68 The Department gives some 
recognition of contaminated emissions in 
the pollution prevention section of its 
sustainable development strategy. The 
strategy established a target requiring 
sewage treatment plants and storm sewer 
discharges to be compatible with 
applicable standards, including provincial 
and municipal standards, by 2000. No 
commitments were made for untreated 
discharge into municipal sanitary sewers. 


13.69 To meet the target for sewage 
treatment plants, the Department is 
committed to completing a Sewage 
Treatment Plant Optimization Program at 
its six plants by 2001. Under the program, 
each plant’s physical design and operating 
procedures are analyzed during an initial 
comprehensive performance evaluation. 
Then, with the assistance of sewage 
treatment experts, the design and 
operating procedures are changed during a 
comprehensive technical assistance phase 
to make the existing plant work at 
optimum performance. 


13.70 = The initial evaluation by National 
Defence of all six departmental sewage 
treatment plants found that they did not 
comply with the applicable legislation or 
departmental policy on effluent quality. 
Most often the causes of non-compliance 
were not the design of the facility but its 
administration and operation. At the time 
of our audit, changes proposed during the 
comprehensive technical assistance (CTA) 


phase were completed at two plants, 
Borden and Trenton, at a cost of 
approximately $350,000 each. By the end 
of the CTA, both met the applicable 
standards. Earlier studies by outside 
consultants had suggested that Borden 
would have to build an entirely new 
facility at an estimated cost of 

$7.6 million and Trenton would have to 
spend an estimated $2.8 million on 
upgrades to bring the plants into 
compliance. The optimization program 
appears to deliver significant savings over 
these earlier cost estimates. 


Liquid effluent is not monitored for all 
contaminants that local standards limit 


13.71 Inspite of this success, however, 
effluent from departmental sewage 
treatment plants is monitored only for the 
contaminants (mainly biological) that the 
plants are designed to remove, and not for 
industrial contaminants such as solvents or 
trace metals (see Exhibit 13.6). 


13.72 Sewage that leaves National 
Defence facilities for treatment at 
municipal sewage treatment plants is not 
subjected to regular testing for all 
contaminants limited by applicable 
standards. Of the six bases we examined 
that discharge sewage to municipal 
facilities, two (Edmonton and 
Wainwright) do no testing. Montreal has 
conducted one-time risk assessment tests, 
which failed to meet applicable standards 
in several respects. Esquimalt and Halifax 
regularly test sewage effluent from areas 
they consider to be high risks. Only 
Bagotville regularly tests its sewage 
effluent, but not for all contaminants on 
which limits are set (see Exhibit 13.7). 
Municipal laws, with which the 
Department does not have to comply, 
generally set limits on a wider range of 
contaminants. Of the bases in our sample 
that do monitor their effluent, all monitor 
for contaminants beyond those included in 
the federal guidelines. However, few of 
them monitor for all contaminants on 
which limits are set in local laws. 
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13.73 Asis the norm elsewhere, storm 
sewers at departmental facilities are 
generally not treated. At the time of our 
audit, effluent leaving National Defence 
facilities in storm sewers was subjected to 
varying degrees of testing (see 

Exhibit 13.8). In addition to federal and 
municipal laws and guidelines, the Air 
force has an effluent monitoring guide that 
requires its facilities to test their storm 
sewer effluent. The three Air force 
facilities we audited monitor storm sewers 
regularly, though not for all contaminants 
limited by applicable standards. 
Bagotville and Cold Lake have found that 
their effluent regularly fails to meet the 
standards set out by the Air Command 
Effluent Monitoring Manual or other 
applicable laws and guidelines. Of the 
other commands, Borden, Edmonton and 
Wainwright do not test storm sewers at all. 
Montreal and Gagetown each recently 
conducted a one-time risk assessment 
study and failed to meet applicable 
standards for several pollutants. Esquimalt 
and Halifax test their storm sewer effluent 


Borden 


Contaminant 
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regularly, but Esquimalt does not test for 
all regulated contaminants. 


13.74 Base staff at Wainwright and 
consultants engaged by Edmonton and 
Gagetown have recommended that the 
regular testing of storm or sanitary sewer 
effluent be increased. This is both to 
ensure compliance with applicable laws 
and policies and, where there is a 
non-compliant emission, to identify the 
source. 


13.75 National Defence should 
conduct an assessment of all liquid 
effluent and air emissions at its bases to 
ensure that the environment is not being 
contaminated by hazardous materials. 
The assessment should be repeated 
periodically, particularly when there 
are major changes to the base 
configuration or activities. 


Department’s response: The Department's 
Sustainable Development Strategy 
includes commitments for storm water and 
sanitary effluents. The actions associated 


Exhibit 13.6 
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Exhibit 13.7 


Monitoring of Effluent Discharged to 
Municipal Sewage Treatment Plants 


Not Tested 


Tested 


Failed at least 20% of tests 


Specific limits set in Guidelines for 
Effluent Quality and Wastewater 


Treatment at Federal 
Establishments, April 1976. 
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with these commitments will be reviewed 
and revised as required. Similar action 
plans will be developed for air emissions. 
The Department undertakes to review and 
revise its existing effluent monitoring 
protocols as required and to develop 
monitoring protocols for air emissions that 


Regular 


conform to current federal regulations and 
guidelines. The Department will continue 
to participate in the interdepartmental 
sub-working group that is addressing 
effluent discharges from federal facilities. 
This sub-working group was established 
under the Federal Committee on 
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Exhibit 13.8 


Monitoring of Storm Sewer Effluent 
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fecal coliform, total coliform, biological oxygen demand and total suspended solids may originate from the 


surrounding communities. 


* Specific limits set in Guidelines for Effluent Quality and Wastewater Treatment at Federal Establishments, April 1976. 
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Environmental Management Systems 


(FCEMS), which has a mandate to identify 


and resolve environmental issues common 
to all federal departments. The 
Department will make a formal request to 
the FCEMS to investigate whether a gap 
exists with respect to air emissions and 
whether this gap should be addressed by 
FCEMS at the national level. 


National Defence participates in 
national emission reporting programs 


13.76 The Department participates in 
national programs that require the 
reporting of hazardous material emissions 
into the environment. The National 
Pollutant Release Inventory (NPRI) is a 
legal requirement under the Canadian 
Environmental Protection Act; it requires 
that use or release of more than 10 tonnes 
be reported for 176 specified chemicals. 
Of these chemicals only one, ethylene 
glycol for aircraft de-icing, is used by 
National Defence in quantities that may 
meet the reporting criterion, depending on 
annual usage. Releases have been reported 
in recent years at Goose Bay and Trenton, 
and at Greenwood in 1994, 


13.77. Accelerated Reduction/Elimin- 
ation of Toxics is a voluntary program 
under which emissions of 117 toxic 
substances are to be reported and reduced. 
Because the Department’s procurement 
and use of hazardous materials is 
decentralized, National Defence is not 
able to report on uses or emissions. 
Instead, it has reduced the range of 
products it uses that contain the targeted 
toxic substances. By October 1998, the 
Department had eliminated from use more 
than 70 percent of the nearly 700 NATO 
stock number items it had identified as 
containing the substances. 


13.78 National Defence has also 
committed to reducing its emissions of 
dichloromethane, used in stripping aircraft 
paint. The commitment is made under the 
Strategic Options Process that entails 
developing mechanisms (“options”) for 


reducing or eliminating chemicals 
declared toxic under the Canadian 
Environmental Protection Act. Air force 
wings have been directed to implement 
the dichloromethane management plan. 


Compliance Management at 
National Headquarters 


Managing for compliance reduces risks 


13.79 Given the persistent compliance 
problems we observed, we looked at the 
management systems intended to ensure 
that the Department meets the legal and 
policy requirements governing hazardous 
materials. A management system 1s a 
process used to provide an organization 
with reasonable assurance that its work is 
conducted in accordance with applicable 
regulatory requirements, professional 
standards and the organization’s own 
policies and procedures. 


13.80 In the environmental field, the 
International Organization for 
Standardization’s (ISO) 14000 series is 
emerging as the voluntary standard for 
management systems. National Defence 
has made a commitment to implement 
environmental management systems that 
are compatible with this standard. 


13.81 We used selected elements from 
the ISO 14004 standards of Environmental 
Management Systems Principles 

(see Appendix) to evaluate the systems the 
Department uses to ensure compliance 
with the legal and policy requirements 
governing hazardous materials. We did 
not audit the entire environmental 
management system at any level in the 
Department. 


National policy has some gaps 


13.82 In December 1997 the 
Department’s sustainable development 
strategy identified hazardous materials as 
one of its five key issues. In 1998, the 
Department reviewed its policy on 
hazardous materials management and 
reissued it in the form of a Defence 
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Administrative Order and Directive 
(DAOD). However, DAODs for 
ammunition management and nuclear 
safety regulation and control have 
remained in draft form for over a year and 
a half. 


13.83, The DAOD on management of 
hazardous materials commits the 
Department to reduce their use, avoid or 
minimize the creation of pollutants and 
wastes, and limit storage to the quantities 
needed for operational requirements. 
Similarly, the draft DAOD on nuclear 
safety commits the Department to protect 
the environment and minimize human 
exposure to ionizing radiation. More 
broadly, several statements in the 
sustainable development strategy, and 
particularly the targets established for 
hazardous materials, commit the 
Department to continually reduce the 
amounts of material used and their 
potential for entering the environment. 
However, as explained in paragraph 13.26, 
the Department’s policy statements on 
hazardous materials management include 
a commitment to meet or exceed 
applicable federal legal and policy 
requirements, but be compatible with 
provincial, municipal and international 
requirements only “where appropriate”. 


13.84 As well, the DAOD on hazardous 
materials management does not define the 
responsibility and accountability of every 
organization involved. It does define the 
responsibilities of various line authorities, 
including users and their supervisors, as 
well as material technical authorities and 
hazardous material control authorities. 
However, unlike the draft DAODs on 
ammunition and radioactive material, it 
does not define the responsibilities of 
functional experts at headquarters, such as 
the Director Supply or Director General 
Environment. Lists of responsibilities can 
be found in some of the hazardous 
materials manuals, but they are not 
necessarily authoritative, complete, or 
up-to-date. 
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13.85 National Defence should define 
the circumstances in which it will apply 
provincial and municipal laws relevant 
to hazardous materials management, 
and should specify the standards that it 
accepts. 


Department’s response: In accordance 
with the environmental management 
system adopted by National Defence, the 
Department is required to review 
environmental legislative and regulatory 
requirements associated with site- and 
event-specific activities. Based on these 
assessments, Bases and Wings will update 
their environmental management systems 
to reflect the provincial and municipal 
regulations and policies that are 
applicable to their location. 


13.86 The responsibility and 
accountability of National Defence 
headquarters organizations involved in 
the management of hazardous materials 
should be defined, documented and kept 
up-to-date. 


Hazardous materials 
management policy 
does not define the 
responsibilities of 
headquarters experts. 


Department’s response: The Department 
will review and revise its internal policy 
on hazardous materials management to 
include the responsibilities and the 
accountability of functional experts 
involved in the management of hazardous 
materials. 


National plans, procedures and 
objectives met our expectations 


13.87 The 1997 sustainable 
development strategy meets the 

ISO 14004 criteria that require an initial 
national environmental review, an action 
plan to carry out policy, and established 
objectives and targets. 


13.88 In his May 1998 Report to the 
House of Commons, the Commissioner of 
the Environment and Sustainable 
Development reviewed the sustainable 
development strategies tabled on behalf of 
28 federal organizations in 

December 1997. The review essentially 
asked whether the sustainable 
development strategies included all the 
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components required by the Auditor 
General Act and A Guide to Green 
Government. The Commissioner 
concluded that in preparing its strategy, 
National Defence did what it had been 
asked to do. 


13.89 In accordance with the ISO 
requirement to define, document and 
update operational processes and 
procedures, a series of top-level 
departmental manuals provide general 
guidance and some additional policy 
requirements in the following areas: 


e hazardous materials procurement, 
storage, packaging, waste disposal, PCB 
handling and spill response; 


e hazardous materials safety, shipping 
and transportation; 


e explosives safety, handling and 
storage; and 


* nuclear safety, use, handling, storage, 
transportation, disposal, decommissioning 
and emergency response. 


13.90 Most of these manuals were 
recently updated or reissued. As well, 
headquarters produces dozens of technical 
publications that define operational 
processes and procedures for handling 
specific hazardous materials in the course 
of performing specific tasks. 


National policies are not effectively 
communicated to the field 


13.91 We found evidence that base and 
unit personnel in some cases were not 
aware of the existence or relevance of the 
national publications; in other cases they 
were not using the latest versions, even 
several months after publication. For 
example, when the 224 people who 
responded to our hazardous products 
sample questionnaire were asked to 
identify policy and procedure documents 
on acquiring hazardous chemical products, 
only 35 — 16 percent — mentioned the 
hazardous materials DAOD, manual, or 
equivalent base or unit procedure. Another 


29 identified other supply manuals or 
procedures that contain elements of the 
hazardous materials acquisition policy and 
procedures. We also observed several 
instances where base or unit documents on 
hazardous materials management that 
were prepared in late 1998 referred to 
national policy documents that had been 
superseded at the beginning of the year. 


National hazardous materials audit 
programs are not complete or consistent 


13.92 Each of the national headquarters 
of the Maritime, Land and Air forces 
operates an environmental audit program. 
The Land program aims to audit every 
base at least every three years. The Air 
force’s aim 1s to audit every wing at least 
every two years. Because the Maritime 
force operates only two bases, its 
environmental audit program aims to 
cover all units every three years. The audit 
programs are based on detailed checklists 
or audit protocols that include sections on 
hazardous materials and on related issues 
such as management systems, air 
emissions, wastewater or stormwater 
emissions, petroleum products and used 
oil. 


13.93 Since the Land and the Maritime 
environmental audit programs were less 
than three years old at the time of our 
audit, not all bases had been audited yet. 
However, their audit cycles were on 
schedule and all audits planned to date 
had been carried out. The initial audit plan 
for the Air force was incomplete since 
only 7 of the 11 wings would be audited 
between 1997, when the revised audit 
protocol was issued, and 1999. The plan 
has since been revised to visit all wings 
between 1998 and 2000. 


13.94 The inspection program for 
licensed radioactive material facilities 
requires biennial inspections by the 
Director General Nuclear Safety. 
Inspection programs for ammunition 
facilities require annual inspections either 
by the Director Ammunition Program 
Management or a qualified explosives 
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inspector from the Chiefs of Staff. 
Checklists are used for these inspections. 


13.95 Both the ammunition and the 
radioactive material inspection programs 
have experienced problems due to a lack 
of resources. In 1998, ammunition storage 
was not inspected at 8 out of 40 facilities. 
Additional inspectors have since been 
hired in an effort to close the gap. The 
radioactive material inspection process 
faces greater difficulty. In 1997 and 1998, 
only 65 percent of the 425 licensed units 
were inspected. Moreover, the 1999-2004 
Business Plan for the Infrastructure and 
Environment group in National Defence 
headquarters has identified a long-term 
inability to oversee the radiation safety 
program and nuclear safety compliance. 
Additional resources have been requested 
to close this gap. 


13.96 We did not audit the quality of 
the audit protocols or the audit and 
inspection reports. However, we did note 
that the protocols are all different. As a 
result, it is not possible to consolidate the 
audit and inspection findings to produce 
meaningful performance information on 
the Department as a whole. 


13.97 Moreover, there are significant 
gaps in the audit programs. The 
headquarters groups responsible for 
personnel and materiel management 
command several field units, including 
Canadian Forces Base Borden, Canadian 
Forces Supply Depots at Edmonton and 
Montreal, and 202 Workshop in Montreal. 
None of these facilities is subjected to 
regular external environmental audit that 
would include assessing hazardous 
materials compliance. 


Performance measurement could 
improve compliance 


13.98 | Weaknesses in the audit and 
inspection programs are compounded by 
the fact that the Department’s sustainable 
development strategy contains limited 
targets and performance measures for 
hazardous materials compliance. While 
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the strategy met the expectations of the 
Commissioner of the Environment by 
including targets that set detailed, clear, 
measurable and time-bound performance 
requirements, the Commissioner’s review 
did not look at whether the targets and 
measures were complete. 


13.99 The strategy’s compliance targets 
cover only fuel storage tanks, sewage 
treatment plant discharges, and storm 
sewer effluent. As we have noted, our 
audit revealed compliance problems 
covering many more aspects of hazardous 
materials management at National 
Defence facilities. As the Commissioner 
noted in his 1998 Report, “Environmental 
performance that is measured, managed 
and rewarded continually improves.” We 
believe that establishing performance 
measures and targets for a broader range 
of compliance requirements and reporting 
on them annually to Parliament would 
provide an incentive to improve the 
current state of compliance. 


There are significant 

gaps in headquarters 
audit and inspection 

programs. 


13.100 Since publication of the 
Department’s sustainable development 
strategy, the interdepartmental Committee 
on Performance Measurement for 
Sustainable Government Operations 
developed draft indicators for 
government-wide reporting. The 
Committee, which includes 
representatives from National Defence, 
was formed in 1997 to focus on how to 
build environmental performance 
reporting systems. Two of the 

four hazardous materials performance 
indicators proposed by the Committee 
focus on compliance (see Exhibit 13.9). 
Using performance indicators to measure 
compliance for more aspects of hazardous 
materials management in the next revision 
of the Department’s sustainable 
development strategy could therefore 
contribute to the development of 
government-wide reporting. As the 
Commissioner of the Environment has 
suggested, using consistent measures 
could allow departments to build on each 
other’s experience and information, and to 
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realize economies of scale in developing 
information systems. 


13.101 Measures of compliance might 
also be an effective way to incorporate 
radioactive materials and ammunition into 
the hazardous materials component of 
National Defence’s sustainable 
development strategy. The Department 
currently holds an inventory of 
ammunition worth more than $3.5 billion, 
and over 130,000 items of equipment with 
a total value of more than $200 million 
that contain controlled amounts of 
radioactive material. We believe that 
inventories as large and as hazardous as 
these cannot be excluded from the 
sustainable development strategy’s 
performance reporting on hazardous 
materials. Because ammunition and 
radioactive materials are operational 
necessities, environmental performance 
measures for them need to demonstrate 
continual improvement in their safe 
management rather than reductions in 
their use. Reporting in these areas should 
be able to make use of the existing facility 
inspection programs and analyses, as well 
as existing processes for accident and 
incident reporting and analysis. 


13.102 In the revision of its sustainable 
development strategy due in 2000, 
National Defence should provide 
performance indicators for hazardous 


Exhibit 13.9 


Proposed Environmental Performance Indicators for 
Hazardous Materials in Sustainable Government Operations 


e number of storage facilities meeting regulations, policies and procedures 


e amount of hazardous waste sent to disposal (by weight/class), timeframe will 
vary 

e number of on-site recycling, treatment, destruction and disposal facilities 
meeting regulations, policies and procedures 


| @ amount of hazardous waste sent to on-site recycling, treatment, destruction 
and disposal 


Source: Federal Committee on Environmental Management Systems, Committee 
on Performance Measurement for Sustainable Government Operations 


materials compliance. The indicators 
should measure continual improvement 
toward a target of full compliance, 
contribute to government-wide 
reporting, and include ammunition and 
radioactive material. 


Department’s response: In the revision of 
its Sustainable Development Strategy due 
in 2000, the Department will include 
appropriate performance indicators for 
hazardous materials by: 


e establishing a departmental 
definition of the term “full compliance”; 


e reviewing and updating, as required, 
the performance indicators for the 
hazardous materials targets included in 
the Department’s current Sustainable 
Development Strategy; 


e continuing to participate in the 
interdepartmental process that is 
developing government-wide performance 
indicators. The Department will include 
government-wide performance targets in 


future Sustainable Development Strategies 


where it is appropriate to do so; and 


e establishing appropriate 
performance indicators for ammunition 
and radioactive materials. 


13.103 To assist in reporting on 
compliance, headquarters audit and 
inspection programs should be 
consistent across the Department, 
should cover all departmental facilities 
and units, and should provide for 
inspecting all facilities and units at least 
every three years. 


Department’s response: The Department 
will review and revise current audit and 
inspection programs, as required, so that 
protocols used at all levels include core 
requirements that must be examined 
during audits and inspections. The 
Department will also establish guidelines 


for the frequency of these audits and 


inspections of its facilities and units. 


13-30 Report of the Auditor General of Canada — September 1999 


Slow implementation of performance 
measures for sustainable development 


13.104 We examined progress to date in 
reporting against the targets and 
performance measures for hazardous 
materials included in the Department’s 
sustainable development strategy. The 
strategy defined six targets and 

13 performance measures; however, we 
did not audit the target for contaminated 
sites or the four related performance 
measures. The Department’s 1998 
Performance Report provided information 
on only one of the remaining targets, 
concerning fuel storage tanks, but not on 
the performance measure associated 

with it. 


13.105 Given the short period of time 
between the tabling of the strategy in 
December 1997 and the March 1998 
cut-off date for the first progress reports, 
we recognize that the Department might 
not have been able to report on all its 
targets. In particular, in December 1997 
there were no precise definitions and 
information mechanisms for the targets of 
reducing the use of high-risk hazardous 
materials and showing downward trends in 
reportable spills and hazardous waste sent 
for disposal. Since then, the Department 
has defined terms and developed reporting 
mechanisms. However, because existing 
data on hazardous waste and reportable 
spills are not consistent, baselines will not 
be established until the new system 
contains data for a full year (March 2000). 
Information for a second year will be 
required before the system can provide the 
first indication of whether the downward 
trend targets are being met. Therefore, 
meaningful information on progress 
toward two of the hazardous materials 
targets will not be available before the 
Department’s fall 2001 Performance 
Report. 
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Year 2000 certification of computer 
systems for hazardous materials is on 
target 


13.106 Ensuring that its mission-critical 
computer systems can accept dates from 
the year 2000 onward is a priority for 
National Defence. As a result, it has 
established a process to confirm the 
readiness of those systems. We reviewed 
the Department’s status reports on 168 
systems that play a role in hazardous 
materials management, including national 
management systems, base-level 
networks, systems that control fuel farms, 
sewage treatment plants and heating 
systems, and computerized munitions. As 
of mid-May 1999, the Department had 
certified 54 systems as compliant. Plans 
were 1n place and on schedule to certify 
the remaining 114 systems by the end of 
September, though 24 had missed some 
deadlines. The Department has 
mechanisms in place to verify a sample of 
systems in order to ensure that procedures 
are followed and technical requirements 
are met. Verification reports had been 
completed on nine of the systems we 
selected. Significant problems were found 
with the procedures followed for two 
systems, but they have been rectified. 


Compliance Management at 
Bases 


Base-level managing for compliance 
could be improved 


13.107 We reviewed the management 
systems at 10 Canadian Forces bases 
against the selected criteria of ISO 14004 
(see Appendix). Management system 
elements were assessed at both the base 
and unit levels. The results are presented 
in Exhibit 13.10. 


13.108 Very few of the compliance 
management components fully met the 
ISO 14004 criteria. The main exception 
was the requirement for an initial 
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Exhibit 13.10 2 
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1. Commitment and Policy 
An organization should define its environmental policy and ensure commitment 
to its EMS. 
e Does the policy embody commitments such as continual improvement; Oo co@ee@egee@eod @ © @ 
monitoring; meeting or exceeding legal requirements? 
e Has an initial environmental review been conducted? @®@eeeeee¢eeese ®@ 
2. Planning 


An organization should formulate a plan to fulfil its environmental policy. 
e Does the plan address legal and policy requirements? 
e Does the plan establish objectives and targets? 


e Is amanagement program established to address objectives and targets; is 
the program reviewed, monitored and revised? 


eS 6: ¢ 
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3. Implementation 

For effective implementation, an organization should develop the capabilities 
| and support mechanisms necessary to achieve its environmental policy, 
objectives and targets. 


e Are responsibility and accountability clearly defined? @®eeedeee@ee8ed80 @ 


e Are reports containing compliance evaluations and opportunities for 
improvement communicated to those responsible? 


| e Are operational processes and procedures defined, documented @eeedqe@eede8de @ 
and updated? 


@ 
© 
@ 
@ 
© 
oO 
@ 
@ 
® 
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4. Measurement and Evaluation 
An organization should measure, monitor and evaluate its environmental 
performance. 
e Does the organization measure and monitor ongoing performance against 
objectives and targets? 
e Does the organization evaluate compliance with legal 
and other requirements? 


| e Does the organization document findings, and corrective 
and preventive actions taken? 
| e Is there a systematic follow-up to ensure the effectiveness 
of preventive actions taken? 
e Are records (such as training activity, inspection activity, details of 
non-conformance) maintained and reviewed? 


e Has the system been audited to determine if it conforms to planned 
arrangements and is properly implemented and maintained? 


O@ 0 6 @ 0 
O88 0686 0 
O @ Oo @ @ 0 
O 8 0 @@ O 
O®@8 0686 0 
O@8@ 0 80 O 
O@ @ @ @ @ 
O®8 0686 0 
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5. Review and Improvement 

An organization should review and continually improve its environmental 
management system, with the objective of improving its overall environmental 
performance. 


e Has the system been reviewed to ensure its continuing suitability and 
effectiveness in order to identify opportunities for improvement? ©. 16) @LOs @.26n-6 168 6-2 | 


e Does the system embody continual improvement by determining root 


causes of non-conformance/deficiencies and developing plans to address @ £6) © 16) 6 104 @ 
root causes? 


O 


Source: Office of the Auditor General @ Fully met @ Partially met © Not met 
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environmental review, which was met in 
most cases by the environmental baseline 
studies conducted at all bases starting in 
the mid-1980s. Department-wide, some 
two thirds of the recommendations from 
those studies were related to hazardous 
materials. 


13.109 The key weakness in compliance 
management is the failure to regard it as a 
continual improvement feedback system. 
While most bases evaluate compliance to 
some degree, there are no established 
objectives and targets, and therefore no 
plan for achieving any. 


13.110 Bases have no performance 
measures for compliance. We have 
already noted that base-level audit and 
inspection programs are highly 
fragmentary. Moreover, the structure for 
compliance management is not reviewed 
to determine whether it is effective and to 
examine the root causes of problems. In 
these circumstances, it is not surprising 
that bases repeatedly display the same 
problems. 


13.111 Although difficult to measure, 
factors contributing to compliance 
management problems include the many 
pressures that the Department and its 
managers have faced in the last five or 
more years. Budgets have been cut and 
devolved, requiring managers to learn new 
skills. Units have moved; bases have been 
closed and others restructured, requiring 
that new procedures be established. 
Personnel levels have fallen, reducing the 
reservoir of functional expertise. 
Particularly at the unit level, the jobs of 
environment officer or hazardous 
materials co-ordinator are assigned as 
secondary duties, perhaps added to other 
secondary duties, while pay has been 
frozen for years at a stretch. In such an 
atmosphere, leadership is vitally important 
in setting goals, providing direction, and 
following up to ensure that the goals and 
direction are carried out. 
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13.112 The potential impact of an 
effective compliance management system 
is indicated by the fact that, at the three 
bases we audited in more detail, we found 
that the more complete the system, the 
better the overall state of compliance. 
Better management systems will protect 
workers and the environment better. 


13.113 National Defence headquarters 
has begun to provide guidance to bases 
and units on developing ISO 14000- 
compatible environmental management 
systems. If successfully implemented, the 
systems should begin to address the many 
weaknesses in compliance management at 
the base level. However, no deadline has 
been established for implementing 
environmental management systems 
across the Department. 


Compliance 
management is not 
implemented as a 
continual improvement 
feedback system. 


13.114 National Defence should ensure, 
as part of implementing environmental 
management systems, that bases and 
units establish performance measures 
for hazardous materials compliance 

that measure continual improvement 
toward a target of full compliance. 


Department’s response: Departmental 
direction with respect to environmental 
management systems will be expanded to 


permit the measurement of continual The Department has 

improvement toward the Department's t set dead f 

definition of “full compliance”. Not set Ceadlines 10f 
implementing 

Some promising developments were ‘ 

hated environmental 


13.115 We noted (see Exhibit 13.10) that | Management systems. 


both Maritime bases in our sample 
(Esquimalt and Halifax) scored in the top 
half of the range of ISO 14004 results, 
while all three Air bases (Bagotville, Cold 
Lake and Trenton) were in the bottom 
half. Results for Land bases (Edmonton, 
Gagetown, Montreal and Wainwright) 
were mixed, while the Personnel Group 
base (Borden) was in the middle. There 
may be opportunities for the Maritime 
bases to share their better practices. 


13.116 Some of the bases we examined 
have combined the various functional 


13-33 


National Defence — Hazardous Materials: Managing Risks to 
Employees and the Environment 


experts responsible for safety and the 
environment (and therefore also for 
hazardous materials) into a single office 
that reports directly to the base 
commander. In Montreal and Bagotville, 
this has resulted in the development of a 
consolidated base-level safety and 
environmental inspection process. All 
base functional experts (safety officer, 
environment officer, hazardous materials 
officer or co-ordinator, radiation safety 
officer, explosives safety officer, fire chief 
and preventative medicine technicians) 
inspect a unit at the same time, using 
checklists. All base units are subjected to 
this “‘multi-disciplinary” inspection once a 
year. The result is an annual “snapshot” of 
each unit’s state of compliance. These 
snapshots are expected to be comparable 
across units and over time, thereby 
providing useful performance information. 


13.117 The Department should require 
the use of a consistent methodology for 
annual environmental audits and 
inspections at the base level. 
Implementing the methodology should 
involve all hazardous materials 
functional experts. 


Department’s response: In the response to 
the recommendation at paragraph 13.103, 
the Department committed to establishing 
core requirements with respect to 
hazardous materials management that 
must be examined during audits and 
inspections programs at all levels. The 
Department will review and revise its 
methodology for audits and inspections of 
Bases and Wings as required so that 


functional experts participate in the audits 


and inspections. The Department will also 
provide guidance on the frequency of these 
audits and inspections at the Base and 
Wing level. 


13.118 We noted another useful initiative 
at Halifax. Acquisition of hazardous 
materials has been restricted to Base 
Supply, except for a list of items 
considered to be low-risk. The approach at 
Halifax goes part way toward initiatives 


introduced in the United States armed 
forces to centralize hazardous materials 
management at the unit or base level. The 
essential features of the American 
programs are: 


e asingle point at the unit or base 
responsible for acquiring, storing and 
dispensing all hazardous materials and 
providing associated safety, health and 
environmental information; 


e an authorization process that 
identifies who is permitted to obtain what 
products; 


e tracking of materials issued to 
individuals, through bar-coding; 


e requiring the return of empty or 
partially used containers before new ones 
are issued; 


e reissue of unused or partially used 
material; and 


e centralized disposal of hazardous 
waste. 


13.119 The American programs have 
reduced hazardous materials acquisition 
costs, waste disposal costs, shop stocks 
and chemical use. According to the UK 
National Audit Office, the Royal Air 
Force has introduced a simplified version 
of the American system that has resulted 
in reduced shop stocks and improved 
waste disposal practices. 


13.120 National Defence should 
conduct a formal assessment or trial of 
a centralized hazardous materials 
management system at the base or unit 
level, aimed at determining: 


e the costs of implementing a 
centralized system; 


e the optimal level of centralization, 
whether at the base, the unit or another 
level; 


e the potential savings in acquisition 
and disposal costs; and 


e the degree of potential 
improvement in compliance. 
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Department’s response: The Department 
commits in principle to conducting a 
formal assessment of a centralized 
hazardous materials management system 
incorporating the requirements outlined in 
the recommendation. As part of this 
assessment, the Department will also 
investigate the centralized hazardous 
materials management systems used by 
other militaries. Trials will be conducted 
on an as-required basis to validate or 
confirm these assessments. 


Choosing Safer Products 


13.121 The most effective means to 
minimize the risks posed by hazardous 
materials is to minimize their use. This is 
also the most effective way to prevent 
pollution. The Department has recognized 
this in its directive on hazardous materials 
management and in its sustainable 
development strategy. 


13.122 The Department is participating 
in a number of voluntary programs to 
reduce its use of hazardous materials, 
including the Accelerated 
Reduction/Elimination of Toxics program 
and the Strategic Options Process. Its 
sustainable development strategy also 
includes a target of eliminating the use of 
specified high-risk hazardous materials. 


13.123 The Department’s approach to 
these programs is to identify items with 
NATO stock numbers in the Canadian 
Forces Supply System that include the 
targeted chemicals. Headquarters staff 
then review the products to determine if 
they are still needed or if there are 
environmentally friendly substitutes 
available. Tools such as the Hazardous 
Material and Preferred Product Selector 
database have been developed to help base 
and unit personnel identify the most 
appropriate substitute for their needs. 


13.124 However, because procurement 
of many hazardous chemical products is 
decentralized, not all products used have 
NATO stock numbers. Approximately 
10 percent or 400 of the chemical 
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products in the Hi-Q information system, 
for example, do not have a stock number, 
a proportion that is likely to increase as 
more products are procured locally. 
Moreover, as a result of local 
procurement, headquarters may be 
unaware of the products used at the local 
level. The Department has recognized this 
problem, and the hazardous materials 
directive therefore makes it mandatory to 
use the materiel information management 
system for life cycle management of 
hazardous materials. However, it appears 
that many personnel are unaware of this 
requirement and rarely comply. Of the 607 
items in our hazardous products sample, 
the personnel responsible for acquiring 
them confirmed that only 247, or 

4] percent, were recorded in the system. 
Lack of information at headquarters may 
mean that products are not included in the 
lists to be assessed and eliminated or 
replaced. 


13.125 The Department should 
strengthen mechanisms for ensuring 
that headquarters is aware of hazardous 
products in use, and that users are 
aware of less hazardous substitutes that 
are available. 


Department’s response: The Department 
has developed a process to replace 
products containing substances of concern 
with more environmentally friendly 
products, through its voluntary 
participation in Environment Canada’s 
Accelerated Reduction/Elimination of 
Toxics program. This process requires that 
restrictions be placed on those products 
containing substances of concern where 
replacements are not possible, are not 
cost-effective or affect operational 
readiness. The Department will develop an 
appropriate communications strategy to 
inform departmental staff at all levels 
about the replacement products, including 
the products that they are replacing. The 
Department also commits to conducting 
annual reviews of the screening 
methodology that it uses to identify 
products containing substances of 
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There are widespread, 
frequent and recurring 
instances of 
non-compliance with 
legal and policy 
requirements 
governing hazardous 
materials. 


concern. The screening methodology will 
be updated as required so those products 
posing the highest risk to workers and the 
environment will be the first priority for 
replacement. 


Conclusion 


13.126 Departmental policies, plans and 
procedures at the national level indicate 
that National Defence is aware of the legal 
framework and best practices for 
hazardous materials management. At the 
base and unit levels, however, 
implementation has been uneven. As a 
result, there are widespread, frequent and 
recurring instances of non-compliance 
with the legal and policy requirements. 
Non-compliance puts workers and the 
environment at increased risk. 
Nonetheless, we did find instances of 
efforts to improve the management of 
hazardous materials in the Department; 
and injury trends, particularly among 
civilian employees, have shown marked 
improvement since 1993. 


13.127 A major factor in the lack of 
compliance is that the existing 
management systems to ensure 
compliance do not promote continual 
improvement. At the base level, objectives 
are not set and hence plans do not exist to 
achieve any. At both the national and base 
levels, audit and inspection programs are 
not consistent or implemented fully, and 
their results do not feed into performance 
measures aimed at continual improvement 
toward full compliance. 
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13.128 Performance measures in 
controlling and reducing emissions are 
also inconsistent, targeting some sources 
but not others. In part, this reflects the fact 
that departmental policy applies standards 
found in provincial and municipal laws 
only “where appropriate’, without 
defining what this means. As a result, the 
Department’s standards are compatible 
with some provincial and municipal laws, 
but not with others. 


13.129 Information and communication 
are problem areas in several aspects of 
hazardous material management. National 
policies do not seem to be communicated 
effectively, and information on hazardous 
materials that are used 1s not readily 
available and communicated. Improved 
information systems or approaches used in 
other countries may offer solutions in 
some areas. 


13.130 National Defence has committed 
itself to implement environmental 
management systems at all levels. If it 
succeeds, the objective to continually 
improve compliance will be embedded in 
policy, plans and procedures at all levels. 
Effective implementation of 
environmental management systems 
would also increase the frequency, 
comprehensiveness and consistency of 
internal audits at all levels, which should 
ensure that external inspections find fewer 
instances of non-compliance. However, no 
deadline has been set for implementing 
environmental management systems 
across the Department. 
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ty About the Audit 


Objectives 


The audit objectives were to determine whether National Defence: 


e manages its use of hazardous materials in a way that minimizes the risks to its employees’ health and 
safety and to the environment; 


e has systems in place that are effective in ensuring continuing compliance with applicable hazardous 
materials management laws, regulations and policies; and 


e has established appropriate sustainable development targets for hazardous materials and can measure its 
performance in meeting them. 


Scope 


Our audit focussed on two management levels: National Defence headquarters, where policies and procedures 
are developed and disseminated; and the base level, where hazardous materials are stored, handled, used and 
disposed of. We looked at policies, procedures and results in the following areas: 


e local acquisition of hazardous material; 

e systems to ensure compliance with legal and policy requirements; 
e audits and inspections; 

¢ programs to monitor air and liquid emissions; 

e records of hazardous materials training; and 


e progress in developing and implementing the Department’s sustainable development strategy objectives 
for hazardous materials. 


Approach 


At the headquarters level, we met with all the organizations in the Department responsible for the various 
types and life-cycle components of hazardous materials. We obtained and reviewed documentation on 
legislation, regulations, policies, audits and management systems. We also discussed implementation of the 
Department’s sustainable development strategy. 


To assess the state of hazardous materials management at the base level, we selected a sample of 10 Canadian 
Forces facilities. Our criteria in selecting the bases were representation of all three services and major types of 
activity, quantities of hazardous material held, number of injuries occurring in recent years, and proximity to 
populated areas. The bases selected were 3 Wing Bagotville, Canadian Forces Base (CFB) Borden, 4 Wing 
Cold Lake, CFB Edmonton, CFB Esquimalt, Combat Training Centre (CTC) Gagetown, CFB Halifax, 
Montreal (Area Support Units Longue Pointe, St-Hubert and St-Jean), 8 Wing Trenton, and Western Area 
Training Centre Wainwright. Reserve force units and outlying detachments supported by the bases were 
excluded. 
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While the bases are not statistically representative, they are significant in their own right. The facilities 
represent almost one third of Canadian Forces bases, stations and depots. They hold almost 60 percent of the 
ammunition and radioactive material recorded in the Canadian Forces Supply System, and almost 80 percent 
of the hazardous chemical products. The bases also employ 35 percent of the Department’s regular force 
military and civilian personnel. The conclusions from this audit should therefore be of interest to 
departmental officials involved in environmental and materiel management. 


At the base level, we met with officials responsible for the environment, hazardous materials, safety, 
preventative medicine, and fire prevention, and with users of the materials. We obtained and reviewed 
documentation on management systems, acquisition, training and effluent monitoring. 


At three bases, CFB Halifax, CTC Gagetown, and 4 Wing Cold Lake, a team led by certified environmental 
auditors conducted audits to determine the current state of compliance. 


To obtain information on policies and procedures for acquiring hazardous materials and bringing them into 
inventory, we selected a sample of hazardous products. We chose 10 types of materials that are widely used 
across the Department, are authorized for local procurement and are among the more hazardous. They 
included specific types of camouflage paint, paint thinner and remover, anti-seize lubricant, propane and acid. 
We obtained copies of all 1,760 inventory listings of hazardous chemical products at the 10 bases. We then 
selected each instance of the materials on the inventory lists and distributed a questionnaire to the person 
responsible for acquiring it. Because the same person was often responsible for acquiring several products for 
each inventory, most people who responded filled out more than one questionnaire. 


e We found 1,096 instances of our 10 target materials on 609 of the 1,760 inventory lists, and distributed 
1,096 questionnaires. 
e We received responses to 734 of the questionnaires. 


e 127 of the responses indicated that the product was no longer used or in inventory, leaving 607 responses 
providing information on processes. 


e The 607 questionnaires were completed by 224 workers. 


To obtain information on training, we requested lists of all personnel who had access to the 609 inventories 
that included our hazardous products sample items. A total of 6,238 personnel were identified as having 
access to those inventories, representing 21 percent of the total personnel on the 10 bases. We then randomly 
selected from among them 50 people per base at 9 of the bases and 65 at Halifax, for a total of 515. Of these, 
61 had recently left the Department or been transferred, or were deployed overseas; therefore, we obtained the 
hazardous materials training records of 454 people. 


Criteria 

We expected that: 

e departmental activities would comply with all requirements for hazardous materials management 
contained in federal legislation and regulations, and in departmental policy; 


e departmental activities would be compatible with provincial acts, regulations, guidelines and municipal 
and international standards, where applied; 


e the Department would have in place management system elements, consistent with the principles of 
ISO 14004, to ensure compliance with legal, regulatory and policy requirements; and 
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e the Department would have identified and implemented hazardous materials management objectives and 
performance indicators that are complete, meaningful, reliable and practical. 


Audit Team 


Assistant Auditor General: David Rattray 
Principal: Peter Kasurak 
Director: Nicholas Swales 


Shaun Bhagrath 
Lori Buck 

Darryl Burgess 
Anne Hardy 
Raymond Kunce 
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Darlene Mulligan 
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Mark Skoczylas 
Robert Stopford 
Kari Swarbrick 


For information, please contact Peter Kasurak. 
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Appendix 


ISO 14004 —- Environmental Management Systems 


ISO 14004 is one of a number of standards published in 1996 by the International Organization for Standardization under 
the 14000 series. Two of the standards, ISO 14001 and 14004, specifically address environmental management systems 
(EMS). 


An EMS is a systematic process that has roots in a tenet of quality management — namely, continual improvement is a 
cyclical process that must address internal and external changes. As defined in the ISO 14000 series, an EMS is “that part 
of the overall management system that includes organizational structure, planning activities, responsibilities, practices, 
procedures, processes and resources for developing, implementing, achieving, reviewing and maintaining the 
environmental policy.” 


Both ISO 14001 and 14004 are process standards, not performance standards. By having the elements of an EMS in 
place, an organization should be better able to manage its environmental concerns, which in turn should lead to better 
environmental performance. 


Whereas ISO 14001 is a specification standard, ISO 14004 is a guidance standard. On the one hand, ISO 14001 identifies 
specific elements that must be met for an organization to seek third-party registration or to self-declare as a 
14001-certified organization. In doing so, an organization has committed itself to meeting all requirements specified in 
the standard. 


On the other hand, ISO 14004 is a guidance document for organizations. It does not identify specific requirements. 
Rather, through the use of descriptions, examples and options, it provides information organizations can use in 
implementing and improving an EMS. 


We used the 14004 standard to assess the capability of National Defence to manage hazardous materials, because it 
provided a more liberal assessment of the existing environmental management framework in place at the Department’s 
facilities. 


There are five main principles under the 14004 standard, with each principle subdivided into numerous elements. We 
selected a number of the elements that are particularly important to compliance and continual improvement. 


The principles and elements are: 


1. Commitment and Policy. An organization should define its environmental policy and ensure commitment to its 
EMS. 


e Does the policy embody commitments such as continual improvement; monitoring; meeting or exceeding 
legal requirements? 


e Has an initial environmental review been conducted? 


i) 


Planning. An organization should formulate a plan to fulfil its environmental policy. 
e Does the plan address legal and policy requirements? 
e Does the plan establish objectives and targets? 


e Is a management program established to address objectives and targets; is the program reviewed, monitored 
and revised? 


3. Implementation. For effective implementation, an organization should develop the capabilities and support 
mechanisms necessary to achieve its environmental policy, objectives and targets. 


e Are responsibility and accountability clearly defined? 
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Are reports containing compliance evaluations and opportunities for improvement communicated to those 
responsible? 


Are operational processes.and procedures defined, documented and updated? 


Measurement and Evaluation. An organization should measure, monitor and evaluate its environmental 


performance. 


Nn 


Does the organization measure and monitor ongoing performance against objectives and targets? 
Does the organization evaluate compliance with legal and other requirements? 

Does the organization document findings, and corrective and preventive actions taken? 

Is there a systematic follow-up to ensure the effectiveness of preventive actions taken? 


Are records (such as training activity, inspection activity, details of non-conformance) maintained and 
reviewed? 


Has the system been audited to determine if it conforms to planned arrangements and is properly 
implemented and maintained? 


Review and Improvement. An organization should review and continually improve its environmental management 


system, with the objective of improving its overall environmental performance. 


Has the system been reviewed to ensure its continuing suitability and effectiveness, in order to identify 
opportunities for improvement? 


Does the system embody continual improvement by determining root causes of non-conformance/ 
deficiencies and developing plans to address root causes? 
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Chapter 14 


National Health Surveillance 


Diseases and Injuries 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 
Accountants. The numbered paragraphs in bold face represent recommendations. 
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Main Points 
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Sound health surveillance information can save lives 
Key players in health surveillance 

Collaboration among all players is essential 

Focus of the audit 


Observations and Recommendations 


Lack of a National Framework for Public Health 


Need for Clearly Defined and Understood Roles and Responsibilities 
Clearly understood roles and responsibilities are key to effective health surveillance 
and disease control 
Participants in the health surveillance process want a more proactive Laboratory Centre 
for Disease Control 


Protocols, Roles and Procedures for Handling Threats to Public Health 
Roles and procedures for controlling diseases at ports of entry are unclear 
Roles and procedures for LCDC in responding to outbreaks of food-borne diseases 
have not been defined 
Roles and responsibilities for dealing with influenza pandemics need to be finalized 


Need for Good-Quality, Comparable Health Surveillance Information 
Lack of formal arrangements for reporting communicable disease data to LCDC 


National Health Surveillance Systems Are Not Working as Intended 
Problems in collecting data on communicable diseases 
Weaknesses in the data on chronic diseases 
National surveillance information on childhood injuries does not yield a complete picture 
Weaknesses in data collection lead to delays in dissemination of surveillance information 


Progress in Filling Surveillance Gaps 
Development of a national diabetes surveillance system has been slow 
A national surveillance system has not yet been developed for heart disease and stroke 


Current Efforts to Improve Health Surveillance 
Work has begun on strengthening Health Protection Branch’s future surveillance capacity 
Pilot projects are under way to improve access to data 
Co-operation is essential to improving health surveillance 


Performance Measurement and Reporting of Results 
Few systems have been evaluated and there are no plans to evaluate others 
Need to develop appropriate indicators for measuring performance 
Only limited use of risk assessment 
Need for improved reporting of results to Parliament 


Conclusion 
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Main Points 


14.1 We observed weaknesses in national surveillance of diseases and injuries that, taken together, have clear 
national implications for public health. First, they compromise Health Canada’s ability to detect, anticipate, 
prevent and control health risks associated with outbreaks of communicable diseases and with other health threats. 
Second, they compromise its ability to plan, carry out and evaluate public health programs and other programs 
that deal with the causes and treatment of diseases. 


Background and other observations 


14,2 Health surveillance is a core function of public health. National surveillance of communicable and 
chronic diseases and of injuries is conducted mainly by Health Canada’s Laboratory Centre for Disease Control 
(LCDC), part of the Health Protection Branch. LCDC depends greatly on its interaction and collaboration with the 
provinces and territories and a variety of other federal departments and non-governmental organizations. 


14.3 There is no specific legislation, policy or agreement that links separate components of public health 
functions at the various levels of government. Indeed, there is a void: current health surveillance activities are 
largely carried out on an ad hoc basis. 


14.4 We found few formal agreements or protocols in place to prevent the entry into Canada of serious 
communicable diseases and to deal with disease outbreaks and threats to public health. The lack of attention to 
formalizing the way these threats are to be managed places the health of Canadians at undue risk. 


14.5 Key surveillance systems that we looked at were not working as intended. For a number of reasons, they 
were not enabling Health Canada to effectively monitor communicable diseases such as influenza, AIDS, 
tuberculosis and food-borne disease. The same is true of systems for surveillance of injuries and chronic diseases 
such as cancer, diabetes, and heart disease and stroke. We also found that performance measurement and reporting 
of results need to be improved. 


14.6 Work has begun on strengthening the Health Protection Branch’s future surveillance capacity. An 
important step in this regard is the Branch’s support of an integrated national surveillance network for public 
health information. It is unclear when such a network will be implemented, and it is paramount that all parties 
involved in this network remain committed to its implementation. 


Health Canada’s responses to our recommendations are included in this chapter. The Department concurs 
with the recommendations and has agreed to take corrective action. In some cases, this action is already 
under way. 
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Introduction 


14.7 Health surveillance is a core 
function of the larger framework for 
public health. That framework links many 
institutions and disciplines to bring 
together scientific and technical 
knowledge in order to promote better 
health and to prevent and control 
communicable and chronic diseases and 
injuries. 


14.8 According to the definition 
adopted by Health Canada, health 
surveillance is “the tracking and 
forecasting of any health event or health 
determinant through the ongoing 
collection of data, the integration, analysis 
and interpretation of those data into 
surveillance products and the 
dissemination of such products to those 
who need to know. Surveillance products 
are produced for a predetermined public 
health purpose or policy objective. In 
order to be considered health surveillance, 
all of the above activities must be carried 
out.” The single characteristic that makes 
health surveillance distinct from other 
health information activities is the 
requirement for a direct link to a public 
health purpose or policy objective. 


14.9 Surveillance at Health Canada is 
undertaken by a number of branches and 
directorates. National surveillance of 
communicable and chronic diseases and 
injuries is conducted mainly by the 
Laboratory Centre for Disease Control 
(LCDC) of the Health Protection Branch; 
it is Canada’s national centre for the 
identification, investigation, prevention 
and control of human disease (see 
Exhibit 14.1). The provinces and 
territories and a variety of other federal 
departments and non-governmental 
organizations provide the data for input to 
LCDC’s systems (Exhibit 14.2). Data 
include provincial and regional statistics 
on hospitalizations for injuries and the 
incidence and prevalence of 
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communicable and chronic diseases, 
physicians’ reports, and so on. 


14.10 National surveillance systems 
produce and disseminate information for 
organizations involved in public health. 
This information is an essential tool that 
helps on a national level to identify, 
investigate, monitor, prevent and control 
diseases and injuries. At the international 
level, national surveillance information 
supports World Health Organization 
surveillance programs. It is also used in 
disease prevention and control efforts with 
other international health organizations 
and centres for disease control in other 
countries. 


14.11 To the extent that the information 
provided by Health Canada’s surveillance 
systems is up-to-date and accurate, it can 
play a valuable role in helping to prevent 
and control diseases and injuries and to 
develop policies and strategies for 
improving the health of Canadians. At the 
same time, incomplete or otherwise 
deficient information can hamper the 
efforts of public health authorities to 
understand and respond appropriately to 
potential health risks. 


Sound health surveillance information 
can save lives 


14.12 The strengths — and any 
weaknesses — in health surveillance 
affect Canadians directly. For example, 
anticipating the timing of a flu epidemic 
and identifying the particular strain of the 
virus involved can save the lives of 


The Laboratory Centre 
for Disease Control is 
Canada’s national 


children and the elderly, who are centre for the 
especially vulnerable to influenza and its identification, 
ensuing complications. Surveillance investigation, 


information is also essential to detecting 
patterns of diseases such as cancer: it can 
be used, for example, to identify possible 
links between the disease and what we eat 
and drink. It can also play a key part in 
looking on a national basis at prevention, 
screening and treatment to determine 
where the “success stories” are — what 
has worked and may have the potential to 
save lives. 


prevention and control 
of human disease. 
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Exhibit 14.1 


Laboratory Centre for Disease Control - Organization and Activities 


In 1999-2000, LCDC has a budget of approximately $43 million and over 270 employees. It is divided into nine bureaus, with staff 
located mainly in Ottawa (with the exception of the Bureau of Microbiology, located in Winnipeg) and over 30 program areas addressing 
various public health issues. 


Bureau Division(s) 
1. Cancer Prevention 
Environmental Risk Assessment and Case Surveillance 
Behavioural Risk Assessment 
Cancer Control Assessment and Surveillance 
2. Cardio-respiratory Cardiovascular Disease 
Diseases and Diabetes Respiratory Disease 
Diabetes 
a | 
3. Reproductive Child Injury 
and Child Health Reproductive Health 
Child Maltreatment 
4. HIV/AIDS, Sexually HIV Epidemiology 
Transmitted Disease HIV/AIDS Surveillance 
and Tuberculosis National HIV/AIDS Laboratories 
Sexually Transmitted Disease Prevention and Control 
Tuberculosis Prevention and Control 
Blood Safety 
5. Infectious Disease Respiratory Diseases 
Enteric, Foodborne and Waterborne Diseases 
Disease Surveillance 
Immunization 
| Bloodborne Pathogens 
Nosocomial and Occupational Infections 
6. Microbiology Bacteriology 
Zoonotics and Level 4 Programs 
National Laboratory for Enteric Pathogens 
Sexually Transmitted Diseases 
Host Genetics and Prions Disease 
Viral Diagnostics 
| Core Services and Business Development 
7. Global Surveillance and Quarantine and Migration Health 
Field Epidemiology Travel Medicine Program 
Field Epidemiology Training Program 
Global Public Health Intelligence Network 
8. Canadian Blood Secretariat 
9. Office of Biosafety Importation and Biosafety 
Biocontainment and Certification 
Biosafety Services 
10. Operations, 
Planning and Policy 
11. Directorate Management/ 
Secretariat 


Source: Laboratory Centre for Disease Control 
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14.13 Health surveillance is critical in 
responding to a changing reality. Global 
factors have dramatically increased the 
danger of communicable diseases being 
transmitted on a worldwide basis. 
Increased travel and migration, global 
food-supply lines, the return of old 
communicable diseases and the 
emergence of new “super bugs” have lent 
anew urgency to surveillance. These 
factors have the potential to pose serious 
threats to the health of all Canadians. 


14.14 Surveillance at work. There are 
examples that show how health 
surveillance has benefited Canadians. For 
example, through a voluntary standard, 
new baby walkers have not been sold in 
Canada since 1989. However, surveillance 
information from Health Canada’s 
Canadian Hospitals Injury Reporting and 
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Prevention Program (CHIRPP) showed 
that in Winnipeg there were still one or 
two babies injured each month in 
accidents involving baby walkers. In April 
1997, acting on CHIRPP information, 
officials launched a one-day blitz in 
Winnipeg to encourage parents to turn in 
any remaining baby walkers. The response 
was immediate, and 168 baby walkers 
were brought to a collection depot. In the 
eight months following the local 
campaign, only four walker injuries were 
seen at the Winnipeg pediatric centre. 


14.15 There have also been cases of 
sudden public health threats to which 
LCDC has had to turn its attention. During 
the recent Kosovo refugee operation, 
Citizenship and Immigration Canada 
(CIC) as the lead department requested 
LCDC’s participation. LCDC was 


Exhibit 14.2 


Key Players in Health Surveillance in Canada 


Laboratory Centre for Disease Control (LCDC) 


e Located in the Health Protection Branch at Health Canada, LCDC is the national centre for the identification, investigation, 
prevention and control of human diseases. Its core activities are national health surveillance, disease prevention and control. 
It relies largely on provincial and territorial surveillance data in order to create a national picture of health risks, patterns and 


trends across Canada. 


Provinces and territories 


e Responsible for health matters within their own jurisdictions. In the case of surveillance and prevention and control of 


diseases, among other things, responsible for: carrying out vital statistics registration; ongoing surveillance of health 
conditions within the province or territory; outbreak investigations within the province or territory; public health 


laboratory-based surveillance; and analysis and interpretation of notifications from physicians, health units and other 


reporting sources. 


Statistics Canada 


e Primarily responsible for the collection and reporting of health, morbidity and mortality statistics. While the Agency is not 
directly involved in surveillance of communicable diseases, it undertakes, often on contract to Health Canada, surveys and 
data integration activities relating to specific chronic diseases. 


Canadian Institute for Health Information (CTH) 


e National, not-for-profit organization working to improve the health of Canadians and the health system by providing quality 
health information. Its mandate is to develop and maintain a comprehensive, nation-wide health information system. CIHI 
provides information that is needed to establish sound health policies, manage the Canadian health system effectively and 
create public awareness of factors affecting good health. 


Non-governmental organizations 


e Numerous coalitions and non-governmental bodies undertake disease surveillance in areas such as cancer, cardiovascular 


diseases and diabetes. 
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A key element in 
national health 
surveillance is 
collaboration — 
between Health 
Canada and the other 
organizations involved 
in public health, and 
among all of them. 


involved in planning and overseeing 
medical aspects of the operation because 
of the health threat associated with this 
mass influx of refugees. Surveillance was 
a key aspect of this operation, identifying 
those with communicable diseases and 
monitoring their movement after their 
arrival. It also helped in estimating the 
potential tuberculosis problems and the 
associated immunization requirements, as 
well as the need to deal with the 
psycho-social problems associated with 
post-trauma stress disorders from which 
many refugees suffered. 


Key players in health surveillance 


14.16 LCDC is only one player — but 
a key one — in the health surveillance 
process. As we have noted, others 

supply the data to LCDC, and federal 
organizations and provincial and territorial 
health systems are responsible for 
implementing any strategies that may 

flow from LCDC’s health surveillance 
activities. 


14.17 Clearly, LCDC depends greatly 
on its interaction with the many. and 
varied players who make a key 
contribution to the effectiveness of the 
health surveillance system. We recognize, 
therefore, that Health Canada’s LCDC is 
not solely responsible for a number of 
observations that we discuss in this 
chapter — in areas such as the quality and 
comparability of surveillance information; 
data collection; and dissemination of 
surveillance information. Nonetheless, 
identified weaknesses need to be fixed, 
and LCDC needs to champion change by 
strongly promoting that all players work 
together to improve the surveillance on 
which Canadians rely to protect their 
health. This chapter raises other issues 
that are within Health Canada’s control — 
in areas such as clarifying roles and 
responsibilities and improving 
performance measurement. These the 
Department could deal with on its own. 


Collaboration among all players is 
essential 


14.18 A key element in national health 
surveillance is collaboration — between 
Health Canada and the other organizations 
involved in public health, and among all 
of them. Such collaboration is necessary 
to agree on and develop common 
procedures and protocols for key 
activities. These activities range from 
collecting and reporting data from 
physicians and other sources (the 
foundation of subsequent efforts to 
identify health risks and ultimately to 
reduce or control them) to responding 
directly to outbreaks of communicable 
diseases. 


14.19 By playing a leadership role in 
fostering greater collaboration among the 
various players — federal, provincial and 
territorial — Health Canada would be 
taking an important step toward improving 
health surveillance and thereby the health 
of Canadians. Regardless of who takes the 
lead, however, this and the other issues 
that we raise need to be addressed without 
delay. 


Focus of the audit 


14.20 We examined the way Health 
Canada carries out national health 
surveillance and control activities, and 
how those activities support the other 
components of the public health 
framework. The audit focussed on 
surveillance and control activities related 
to communicable and non-communicable 
diseases as well as to injuries. We 
examined the Department’s processes for 
collecting, analyzing and interpreting data 
and disseminating information, and how 
that information has been applied to 
prevention and control activities. Our 
examination also looked at ongoing 
“transition” initiatives of the Health 
Protection Branch that affect national 
health surveillance. Further details on the 
audit scope, objectives and criteria are 
presented at the end of the chapter in 
About the Audit. 
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Observations and 
Recommendations 


Lack of a National Framework for 
Public Health 


14.21. The provinces and territories are 
responsible for providing many public 
health services; Health Canada is 
responsible for protecting Canadians 
against risks to health and the spread of 
diseases. However, there is no specific 
legislation, policy, or agreement that links 
separate components of public health 
functions at the various levels of 
government. 


14.22. Each province and territory has 
its own public health legislation covering 
the responsibilities of medical officers of 
health with respect to public health, 
surveillance and disease control. Their 
responsibilities include detecting and 
investigating cases of communicable 
diseases, and their powers extend to taking 
the necessary steps to limit the spread of 
diseases. 


14.23. For example, in Alberta, 
provincial legislation requires a medical 
officer to investigate all cases of 
communicable diseases that must, by law, 
be reported in the province. The 
investigation is to establish the cause of 
the disease, the mode of transmission and 
the probable source, and to identify others 
who may be at risk. The medical officer 
takes whatever steps are reasonably 
possible to treat diseases in those who 
may already have been infected, protect 
those who have not already been exposed, 
break the chain of transmission and 
prevent the spread of the disease, and 
remove the source of infection. 


14.24 While each province and territory 
has a legislated public health function, at 
the national level there is no formal public 
health function established that links the 
separate components in the provinces and 
territories. Instead, there is an informal 
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system that relies on personal contacts 
rather than formal arrangements. As we 
note later in the chapter, there is a void; 
current health surveillance activities are 
largely carried out on an ad hoc basis. 


14.25. Health Canada should initiate 
discussions with provinces and 
territories on a national framework that 
would link separate components of 
public health functions at the various 
levels of government to protect 
Canadians against risks to health and 
the spreading of diseases. 


on an ad hoc basis. 


Department’s response: Agreed; 
implementation is under way. Most 
recently, funding has been renewed and 
increased for the National Health 
Surveillance Infostructure (NHS1), the 
federal component of the Network for 
Health Surveillance in Canada now being 
developed, and planning is well advanced 
for the second phase. By the end of the 
year, the federal/provincial/territorial 
Working Group on Health Surveillance 
will prepare for the consideration of 
deputy ministers of health a strategic plan 
and a business plan for the development of 
the Network. In addition, surveillance is 
being considered in the context of Health 
Canada’s review of federal health 
protection legislation. 


Need for Clearly Defined and 
Understood Roles and 
Responsibilities 


Clearly understood roles and 
responsibilities are key to effective 
health surveillance and disease control 


14.26 Globalization has created an 
environment for disease and for its 
transmission that never existed before. 
Health concerns are no longer always the 
responsibility of one jurisdiction, as 
diseases do not respect borders. This 
means that collaboration and co-operation 
between jurisdictions are vital to effective 
surveillance. 


14.27. ~— Given that national health 
surveillance and disease control involve 
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Current national health 
surveillance activities 
are largely carried out 


14-1] 


National Health Surveillance: Diseases and Injuries 


Formal arrangements 
are important to clarify 
roles, responsibilities 
and procedures of all 
players so that 
authorities can act 
quickly when public 
health is at risk. 


many jurisdictions and players, all those 
involved must clearly understand their 
respective roles and responsibilities. This 
is particularly important in responding to 
emergency situations and disease 
outbreaks. 


14.28 LCDC is one of the key 
participants involved in national health 
surveillance and in investigating and 
controlling disease outbreaks. 
Accordingly, we would expect that other 
players in the health surveillance process 
would be aware of LCDC’s central role 
and responsibilities in these areas. 
However, we found that Health Canada 
has not clearly communicated LCDC’s 
role and responsibilities to provincial and 
local health authorities. Not surprisingly, 
they do not fully understand the part that 
LCDC plays in health surveillance and in 
the prevention and control of diseases. 


Participants in the health surveillance 
process want a more proactive 
Laboratory Centre for Disease Control 


14.29 In November 1997, LCDC 
surveyed medical officers of health, 
epidemiologists, public health staff and 
laboratory directors at the provincial, 
regional and local levels. The survey 
found that participants would like LCDC 
to play a more proactive role in health 
surveillance. Some local practitioners 
indicated that LCDC’s role is largely 
invisible; others were unaware of the 
surveillance-related support it offers 
beyond high-profile activities such as 
conferences and publications. 


14.30 ~The survey also noted that 
LCDC’s complex and ever-changing 
organizational structure is confusing to 
provincial and territorial participants. 
Often they are not sure whom they should 
call when they need advice or other 
support from LCDC. As well, the survey 
indicated that the provinces and territories 
strongly agree on the need for LCDC to 
provide leadership by carrying out 
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national surveillance work and providing 
analytical support. 


14.31 As part of its transition initiative, 
the Health Protection Branch has 
responded to this concern by initiating 
efforts to clarify roles and responsibilities 
within an integrated national health 
surveillance network. These efforts 
include consultations with stakeholders 
across Canada. 


Protocols, Roles and Procedures 
for Handling Threats to Public 
Health 


14.32 The success of national health 
surveillance depends largely on the ability 
of federal, provincial, territorial and local 
health departments and organizations to 
develop partnerships, to collaborate and to 
understand clearly what needs to be done 
to prevent diseases and mitigate their 
impacts — who should do it, and when 
and how it should be done. Where 
jurisdiction is shared, formal arrangements 
are important to clarify roles, 
responsibilities and procedures of all 
players so that authorities can act quickly 
when public health is at risk. 


14.33 We expected that formal 
agreements or protocols would be in place 
to prevent the entry into Canada of serious 
communicable diseases such as 
drug-resistant tuberculosis, and to deal 
with disease outbreaks and threats to 
public health. However, we found that 
there are few formal agreements or 
protocols for dealing with these potential 
problems. In our view, the lack of 
attention to formalizing the way these 
threats are to be managed places the 
health of Canadians at undue risk. 


Roles and procedures for controlling 
diseases at ports of entry are unclear 


14.34 Globalization and ever-increasing 
international travel provide the 
opportunity for various communicable 
diseases or infected individuals to enter 
Canada. In 1978, Health Canada published 
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a Canadian contingency plan for handling 
viral hemorrhagic fevers and other 
unusual imported communicable diseases. 
The purpose of the contingency plan is 
twofold. First, it defines a co-ordinated 
response to the importation of suspected 
and confirmed cases of unusual or 
emerging communicable diseases. Second, 
it provides guidance on dealing with 
infected or potentially infected people and 
those with whom they have come in 
contact. The goal is to prevent diseases 
from being introduced into Canada and 
spread. 


14.35 = Although this plan was revised in 
1985 and again in 1997, it still does not 
clearly define roles and procedures for 
those involved in this work — that is, the 
plan does not specify who is responsible 
for doing what. We found that 
contingency plans and protocols with 
clearly defined roles and procedures exist 
at ports of entry only in British Columbia 
and at Lester B. Pearson International 
Airport in Toronto. At the time of our 
audit, Health Canada officials expected 
that a protocol at Montreal International 
Airport, Dorval would be finalized in the 
near future. No protocols are in place at 
other ports of entry. 


14.36 An incident involving a suspected 
case of a highly contagious disease 
occurred at Montreal International 

Airport, Dorval in August 1998. The case 
illustrates the importance of having 

clearly defined protocols to deal with such 
incidents. 


14.37 In this case, Customs detained an 
arriving passenger suspected of being 
infected; the rest of the passengers were 
released. Federal quarantine officials were 
not consulted about either the initial 
detention of the individual or the release 
of the other passengers. The individual 
was eventually examined by Health 
Canada officials. If the individual 
suspected of having the disease had 
actually been infected, it would have been 
difficult to trace all the passengers who 
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had been on the flight, in an effort to 
avoid an outbreak. At the time, there was 
no specific protocol at this airport that 
clearly specified roles and procedures for 
dealing with such incidents. Nor was there 
a formal memorandum of understanding 
between Health Canada and Revenue 
Canada (Customs) on how to manage this 
kind of incident. At the airport, there was 
significant confusion about who had what 
authority to detain or release passengers. 
Most of the federal and local officials who 
were involved (including officials from 
Revenue Canada and Citizenship and 
Immigration Canada) did not know that a 
Canadian contingency plan for viral 
hemorrhagic fevers and other related 
diseases even existed. Immediately after 
this incident, Revenue Canada issued 
interim instructions to its officials for 
quarantine situations. Exhibit 14.3 
provides details of the incident. 


Roles and procedures for LCDC in 
responding to outbreaks of food-borne 
diseases have not been defined 


14.38 We noted that the only 
documented, formal arrangements 
concerning food safety matters were three 
agreements. In addition to a 1996 
agreement between Health Canada, the 
Department of Fisheries and Oceans and 
Agriculture and Agri-Food Canada, in 
1997 Health Canada and the Canadian 
Food Inspection Agency signed a 
Memorandum of Understanding dealing 
with food safety and related areas. The 
Canadian Food Inspection Agency and the 
Food Directorate of the Health Protection 
Branch also signed an MOU dealing with 
food safety emergency response. It gave 
Health Canada responsibility for 
conducting health hazard evaluations at 
the request of the Canadian Food 
Inspection Agency. However, there is no 
agreement that specifically covers the role 
of LCDC in the investigation of 
food-borne disease. 


14.39 Nor do formal protocols exist 
between LCDC and the provinces and 
territories for investigating and responding 
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to interprovincial or national outbreaks of 14.40 LCDC prepared a draft 
food-borne disease. Such protocols are emergency response plan for internal use 
important because many food products are _ in August 1997 that described all aspects 
so widely distributed that large numbers of _ of responses to various general 

people can become infected within a short — emergencies (including outbreaks of 


time. At the time of our audit, an communicable diseases). The plan 
investigation protocol was being drafted directed that all LCDC employees be 
among Health Canada, the Canadian Food __ responsible for both familiarizing 
Inspection Agency and provincial/terri- themselves with the plan and following 
torial governments, but it had not been the policies and procedures it outlined. We 
finalized. did not see any evidence that departmental 


Exhibit 14.3 


Responding to a Suspected Entry of a Dangerous Disease Into Canada 


Background 


e On 27 August 1998, Revenue Canada’s Customs Intelligence received an anonymous call that an individual who would be 
entering Canada was infected with a dangerous disease and in possession of contraband narcotics. The information was 
considered to be “soft”, but a “lookout” was issued to intercept this person on entry. 


e On the evening of 30 August 1998, the individual arrived at the Montreal International Airport, Dorval from New York City 
and was detained at the airport in one of Customs search rooms. A local ambulance was called to transport the passenger to a 
hospital, but the attendants were hesitant to move the person. Someone also suggested isolating 15 staff who had had close 
contact with the individual after arrival. Other passengers were released after the airline obtained information on how to 
locate them should follow-up be required. Federal quarantine officials were not consulted. The plane then departed with a new 
cohort of passengers to New York City. 


e Health Canada’s Laboratory Centre for Disease Control (LCDC) was notified that evening. It contacted Customs officials, the 
federal stand-by quarantine officer in Montreal, the La Guardia Airport authorities in New York and the U.S. Centers for 
Disease Control and Prevention (CDC). CDC had not been aware of the situation since its on-site quarantine officers were not 
consulted. 


e LCDC officials proceeded immediately to the airport to interview the individual (a returning Canadian) and review his travel 
documents. The individual was released soon after LCDC quarantine authorities determined that he was not infected with a 
dangerous virus. 


Issues 


e There was a significant breakdown in communication, and confusion as to who had what authority, in responding to this 
suspected dangerous disease incident. 


e On31 August 1998, LCDC officials attended a debriefing session with representatives of the Airport authority, the airline, 
RCMP, Revenue Canada, Citizenship and Immigration Canada and the local ambulance. Except for the RCMP, none was 
aware that there was a Canadian contingency plan for viral hemorrhagic fever and other dangerous communicable diseases, 
published by Health Canada. 


| ¢ Revenue Canada’s Customs Intelligence received the anonymous call on 27 August 1998, four days before the individual was 
identified and detained. However, it did not follow its procedure for notifying Health Canada when a health risk is involved. 
Had advance notice been given to LCDC, it could have worked out a plan in collaboration with the Airport authority to 
identify respective responsibilities and the necessary action if and when the individual was detained. Such action might have 
prevented the unnecessary delays and anxieties caused at the airport. 


e There was no formal memorandum of understanding between Health Canada and Revenue Canada on the management of 
persons seeking admission to Canada with a suspected quarantinable disease. At the time of the incident, there were no formal 
protocols at this airport with respect to suspected importation of dangerous communicable diseases. It was unclear who had 
what authority. All the other passengers on that plane were released without consulting federal quarantine authorities. If the 


individual had been carrying the suspected virus, it would have been difficult to trace all the passengers who had been aboard 
the flight. 
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officials had consulted the plan in 1998 
when a national outbreak of food-borne 
disease posed serious risks to many 
children. In fact, those directly responsible 
for managing the outbreak had never seen 
the plan. This outbreak of food-borne 
disease is discussed in Chapter 15 of this 
Report. 


Roles and responsibilities for dealing 
with influenza pandemics need to be 
finalized 


14.41 In 1996, LCDC co-ordinated the 
drafting of a contingency plan for 
pandemic influenza. The plan assigns 
Health Canada responsibility for 
co-ordinating its implementation in the 
event of a pandemic. The plan provides a 
framework for planning, preparing and 
implementing a response to a pandemic. It 
identifies the roles of a pandemic 
influenza committee, as well as federal 
departments, provinces and designated 
professional organizations. It also includes 
estimates of vaccine requirements for 
anticipated high-risk groups as well as the 
capacity of manufacturers for expanded 
production. 


14.42 In June 1998, federal, provincial 
and territorial deputy ministers of health 
indicated that they wanted to develop a 
memorandum of understanding regarding 
their respective roles and responsibilities 
in the event cf an influenza pandemic, and 
to agree on the creation of a pandemic 
influenza committee. As of May 1999, 
this memorandum of understanding was 
still being developed. 


14.43 The last pandemic was in 1968. 
Historical trends and available scientific 
knowledge have led health officials to 
predict that another pandemic is likely. 
New strains of influenza are appearing 
that threaten the health of Canadians. It is 
important, therefore, that formal plans and 
agreements be in place to deal with 
another pandemic. 


14.44 Health Canada should finalize 
agreements and protocols with relevant 
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organizations covering roles, 
responsibilities and procedures for 
handling threats to public health, such 
as controlling diseases at ports of entry, 
managing outbreaks of food-borne 
diseases and dealing with influenza 
pandemics. 


Department’s response: Agreed. Health 
Canada is collaborating with the 
Canadian Food Inspection Agency (CFIA) 
and provincial and territorial governments 
to finalize a protocol specifically intended 
for managing food-borne illness 
outbreaks. The development of further 
such agreements with relevant 
organizations, including those for disease 
control at ports of entry, will ensue. 


A memorandum of understanding on 

a national response to an influenza 
pandemic is under intensive discussion 
with the provinces and territories at 
this time. 


Need for Good-Quality, 
Comparable Health Surveillance 
Information 


14.45 We would expect that all players 
in the health surveillance process would 
use common standards and protocols to 
classify, collect and report data on 
diseases and injuries. Common or uniform 
standards and protocols are critical to 
ensuring that surveillance information is 
comparable across the country. 
Information needs to be comparable if 
governments, decision makers and the 
public are to be properly informed about 
risk patterns and trends in the occurrence 
of diseases and injuries. 


Lack of formal arrangements for 
reporting communicable disease data to 
LCDC 


14.46 Health Canada has no formal 
arrangements with the provinces covering 
the procedures for collecting and 
exchanging data on communicable 
diseases. It drafted a memorandum of 
understanding covering the exchange of 
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data on these diseases some 10 years ago, 
but this was never finalized with the 
provinces and territories. Currently, 
provinces and territories report cases of 
nationally reportable communicable 
diseases to LCDC on a solely voluntary 
basis, and they submit the data according 
to different criteria. For example, 
information on tuberculosis that LCDC 
receives (and then presents) is based on 
the date of onset of illness in Ontario but 
the date of diagnosis in all other 
provinces. This makes it difficult to 
compile a national picture of how many 
people have tuberculosis and for how long 
they have been infected. 


14.47, LCDC has developed forms for 
collecting data on tuberculosis and AIDS 
as well as for several other diseases. These 
forms provide a structure for the provinces 
and territories to follow when submitting 
their data on these diseases. We note that 
to encourage physicians to report their 
AIDS data, British Columbia has issued a 
“short form” version of LCDC’s form. 
This means that, even though it has 
specified its requirements, LCDC is 
unable to collect all the data it needs. Nor 
have similar forms been developed for 
most diseases. As a result, provinces 
submit widely varying types of data to 
LCDC, depending largely on what data 
they have available or choose to send. 
LCDC receives large amounts of data, but 
much of the data received is of limited 
usefulness because they are not 
comparable or uniform. For example, we 
found that provinces submit case-by-case 
data on a total of 114 elements. However, 
of these elements, only eight are common 
to — that is, reported for — all cases. We 
also found that no common format exists 
for reporting data. LCDC receives some 
data in electronic format and other data in 
hard copy. 


14.48 Co-ordinating these disparate 
reports at the federal level is a 
time-consuming undertaking. 
Non-comparable data and different 
reporting procedures make it difficult to 
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produce timely reports that yield accurate 
national “snapshots” of individual 
communicable diseases. 


14.49 = The national Advisory 
Committee on Epidemiology (ACE), 
which includes representatives from the 
provinces and territories, has helped to 
compile definitions of communicable 
diseases and to classify certain ones as 
“notifiable”. Notifiable diseases are those 
that are reported across Canada (see 
Exhibit 14.4). However, provincial and 
territorial health authorities are not 
obliged to use ACE’s definitions and its 
list of notifiable (reportable) diseases. 
Furthermore, there is still a need for 
agreement on which of several emerging 
diseases should be added to the list of 
reportable diseases and how they are to be 
defined. 


14.50 For example, chicken pox was 
reported nationally between 1924 and 
1959, and then it was dropped from the 
list of reportable diseases. In 1986, it was 
put on the list again. Because new 
vaccines are available for chicken pox, 
surveillance information is important to 
help determine their efficacy. However, 
since 1986, reporting of chicken pox to 
LCDC by the provinces and territories has 
not been universal. Incidence data from 
1998 are available for only eight 
provinces and territories, representing 
only about 55 percent of Canada’s 
population. It is estimated that of the 
number of cases projected annually — 
approximately 380,000 — fewer than 

20 percent are reported. 


14.51 Clearly, comparable surveillance 
data are essential to estimate the size of a 
health problem and to determine its 
economic burden on society, to 
characterize trends, and to evaluate 
intervention and prevention programs. 
Deficiencies in our national health 
surveillance information also affect Health 
Canada’s ability to provide valid 
information for use internationally to 
address global issues of disease control. 
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14.52 Health Canada should work 
with provinces and territories to 
establish common standards and 
protocols for classifying, collecting and 
reporting data on communicable 
diseases. 


Department’s response: Agreed; 
implementation is under way. In 
conjunction with the Communicable 
Diseases Subcommittee of its Advisory 
Committee on Epidemiology, which 
comprises representatives from all 
provincial and territorial ministries of 
health, the Laboratory Centre for Disease 
Control has concluded an agreement on 
which diseases are nationally notifiable 
and on definitions of surveillance. The 
next task, scheduled to commence in the 
fall of 1999, will be to develop common 
data standards for the reporting of 
nationally notifiable diseases. 


National Health Surveillance 
Systems Are Not Working as 
Intended 


14.53. A critical component of health 
surveillance is information. To carry out 
its surveillance responsibilities, Health 
Canada needs health surveillance systems 
that allow it to monitor diseases and 
injuries effectively on a national level. 
These systems are central to collecting, 
analyzing and disseminating information 
on the incidence and prevalence of an 
injury or disease (the number of new cases 
and the total number of cases of a given 
disease, and who — age, gender, 
occupation, etc. — is most likely to suffer 
from it). 


14.54 Good-quality and timely 
information 1s the critical element in 
anticipating, preventing and controlling 
health risks that affect the well-being of 
Canadians. The purpose of Health 
Canada’s surveillance systems 1s to 
provide such information. 
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Exhibit 14.4 


Nationally Notifiable Diseases! 


Disease » Year First Reported 
AIDS 1986 

Amoebiasis 1927 

Botulism 1933 | 
Brucellosis 1928 

Campylobacter 1986 

Chancroid 1979 


Chlamydia, genital 
Chicken pox 

Cholera 

Diphtheria 

Giardiasis 

Gonococcal infections 
Gonococcal ophthalmia neonatorum 
Heamophilus influenzae B 
Hepatitis A 

Hepatitis B 

Hepatitis C 

Hepatitis non-A, non-B 
Legionellosis 

Leprosy 

Listeriosis 

Malaria 

Measles 

Meningitis, pneumococcal 
Meningitis, other 
Meningitis, viral 
Meningococcal infections 
Mumps 

Paratyphoid 

Pertussis 

Plague 

Poliomyelitis 

Rabies 

Rubella 

Rubella, congenital 
Salmonellosis 

Shigellosis 

Syphilis. congenital 
Syphilis, early latent 
Syphilis, early symptomatic 
Syphilis, other 

Tetanus 

Tuberculosis 

Trichinosis 

Typhoid 

Verotoxigenic Escherichia coli 
Yellow fever 


1990 
1924-1959, 1986 
1974 

1924 

1983 

1924 

1979 

1979 
1927-1958, 1969 
1969 

1991 

1983 | 
1986 
1925 
1990 
1929 
1924 
1979 

1979 

1952 

1924 
1924-1959, 1986 
1924-1952, 1969 
1924 

19243 

1924 

1927 

1924 

1979 
1958 | 
1924 

1992 

1992 

1979 

1924 

1957 

1924 

1929 
1924-1952, 1969 
1990 

19243 


! Provinces and territories are not required by legislation to report on this list of 
diseases to LCDC. 

2 Where reporting stopped, for whatever reason, the dates indicate for what period 
and then the year when reporting began again. 


3 Reportable since 1924, but no cases have been reported. 


Source: Laboratory Centre for Disease Control 


Report of the Auditor General of Canada — September 1999 


14-17 


National Health Surveillance: Diseases and Injuries 


Taken together, the 
weaknesses that we 
observed have clear 
national implications 
for public health. 


14.55 Key surveillance systems that we 
looked at were not working as intended. 
For a number of reasons, they were not 
enabling Health Canada to effectively 
monitor communicable diseases such as 
influenza, AIDS, tuberculosis and 
food-borne disease. The same is true of 
systems related to chronic diseases such as 
cancer, diabetes, and heart disease and 
stroke. 


14.56 Taken together, the weaknesses 
that we observed have clear national 
implications for public health. First, they 
compromise Health Canada’s ability to 
detect, anticipate, prevent and control 
health risks associated with outbreaks of 
communicable diseases. Second, they 
compromise its ability to plan, carry out 
and evaluate public health programs and 
other programs that deal with the causes 
and treatment of diseases. 


Problems in collecting data on 
communicable diseases 


14.57 The lack of common standards 
and agreed-upon procedures for reporting 
information at the provincial and 
territorial levels has a significant effect on 
systems at the national level. 


14.58 We recognize that Health Canada 
does not have direct control over the 
quality and completeness of the data that 
others submit. Nevertheless, any 
deficiencies in these data severely impair 
the effectiveness of the Department’s 
national health surveillance activities. 


14.59 We found widespread delays in 
reporting cases of AIDS, tuberculosis, and 
other communicable diseases to Health 
Canada. The Department allows provinces 
and territories eight weeks after the end of 
each six-month period to report all AIDS 
cases. This timing permits it to publish a 
surveillance report within one month after 
receiving the data. However, we found 
that about half of the recorded AIDS cases 
were not reported until nine or more 
months after diagnosis. The longest delay 
in reporting was 16 years. Moreover, some 
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physicians do not report all AIDS cases 
(the extent of this non-reporting is also 
unknown). 


14.60 Because of these delays in 
reporting and the unknown extent of 
non-reporting, the AIDS-related 
information that Health Canada produces 
does not accurately portray, for example, 
the number of people living with AIDS, or 
the number who have died from it. Such 
basic information is essential to 
developing effective national strategies 
and programs to deal with this disease. 
The Department has recognized these 
problems for a long time. Since 1993, it 
has attempted to apply statistical methods 
to adjust the number of AIDS cases in 
order to take reporting delays into 
account, and it has included the analysis 
in its surveillance reports as a regular 
feature. However, it now suspects that 
reporting patterns may have changed in 
the past year or two and that these changes 
may need to be incorporated in the 
statistical methods it uses to analyze 
reporting delays. The Department believes 
it is necessary to assess the reporting 
patterns and improve the methods before 
new adjustments are published. Therefore, 
it did not include this feature in its report 
for 1998. Nevertheless, AIDS cases 
diagnosed by physicians but never 
reported to LCDC cannot be estimated by 
this analysis. 


14.61 We recognize that Health Canada 
has no authority to compel physicians to 
report communicable diseases. 
Non-reporting is largely beyond its 
control, since authority rests with the 
provinces. The 1997 report of the Krever 
Commission recommended that the 
governing bodies of physicians and 
surgeons in the provinces and territories 
require physicians to report communicable 
diseases. However, provincial and 
territorial practitioners continue to either 
underreport communicable diseases or not 
report them at all. This continues to be an 
important, long-standing problem. 
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14.62 We found weaknesses in the data 
being input to FluWatch, a system for both 
collecting “local” data on influenza and 
feeding the results back to the provinces 
and territories to provide them with 
national influenza-related information. 
Regional reporting varied widely, with 
some regions not reporting at all. In 
1997-98, 191 physicians were recruited 
from across Canada to provide weekly 
reports on influenza cases over the 
reporting period. On average, only 115 
physicians (60 percent) submitted a report 
each week, and only 100 (52 percent) 
submitted reports for at least 20 of the 27 
weeks required. Moreover, the 
Department does not obtain timely and 
accurate data on hospitalizations and 
deaths from pneumonia and influenza for 
its FluWatch report. This information is 
essential for developing a strategy to 
ensure that the most vulnerable in the 
population (the elderly and children) will 
be vaccinated in the event of an influenza 
pandemic. 


14.63 Health Canada should work 
with provinces and territories to explore 
ways to improve the reporting of 
communicable diseases by physicians. 


Department’s response: Agreed. Health 
Canada will commence discussion of this 
important issue at the next meeting of the 
Council of Chief Medical Officers, to be 
held this fall. 


Weaknesses in the data on chronic 
diseases 


14.64 As was true of data collection 
relating to communicable diseases, there 
are gaps in surveillance information on 
certain chronic diseases — for example, 
diabetes and heart disease and stroke. 
Some work is being done to correct this 
situation (see paragraphs 14.81 to 14.88), 
but further work is needed to generate 
surveillance information on these chronic 
diseases. 


14.65 We reviewed two cancer 
surveillance systems: the Childhood 
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Cancer Treatment and Outcome 
Surveillance System and the National 
Enhanced Cancer Surveillance System. 
We noted weaknesses similar to those we 
found in the data on communicable 
diseases. Here, too, deficiencies in the 
data have meant important gaps in 
information on cancer prevention, control 
and treatment outcomes at the national 
level. 


14.66 The Childhood Cancer Treatment 
and Outcome Surveillance System 1s 
designed to provide information on access 
to treatment, outcomes, and regional 
differences in diagnosis and initial 
therapy. We found that Health Canada is 
able to obtain only partial data for 1995 
and 1996 from Ontario, which accounts 
for a significant proportion of childhood 
cancer cases in Canada. We also noted 
that the system contains incomplete data 
on youths between 15 and 19 years of age. 
Individuals in this age group often receive 
treatment at general hospitals, which, 
unlike pediatric hospitals, do not typically 
supply data to the system. 


14.67 The National Enhanced Cancer 
Surveillance System has the potential to 
provide comprehensive information on the 
linkages between cancer and 
environmental factors. However, the 
Department told us that there are gaps in 
environmental data because complete data 
are not available from most provinces. 
Some surveillance information has been 
produced, but gaps need to be filled if the 
system is to reach its full potential. 


14.68 Health Canada, in 
collaboration with provinces and 
territories and with other organizations, 
should strengthen its Childhood Cancer 
Treatment and Outcome Surveillance 
System and National Enhanced Cancer 
Surveillance System to ensure that gaps 
in surveillance information are filled. 


There are gaps in 
surveillance 
information on certain 
chronic diseases. 


Department’s response: Agreed; 
implementation is under way. The 
Laboratory Centre for Disease Control's 
review of the sources of data used for all 
of its surveillance systems will be 
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completed in September 1999. This will be 
followed by a systematic evaluation of the 
effectiveness of its surveillance activities, 
including a review of the quality of data 
sources. The Department is currently 
assessing its needs for cancer 
surveillance, particularly related to the 
environment. Subsequent improvements in 
data sources and enhancement of data 
quality where necessary will assist in 
filling gaps in surveillance information in 
all systems. 


National surveillance information on 
childhood injuries does not yield a 
complete picture 


14.69 In 1990, LCDC’s Child Injury 
Division established the Canadian 
Hospitals Injury Reporting and Prevention 
Program (CHIRPP). This is a surveillance 
system for collecting data on the nature 
and circumstances of injury of persons 
seeking emergency medical care in 
Canadian hospitals. CHIRPP collects 
injury statistics from 16 hospitals, 
including 10 pediatric hospitals. Its focus 
is on children, and its objective is to 
reduce the incidence and severity of 
childhood injuries from all causes. 


14.70 CHIRPP is an important initiative 
that provides useful information. 
However, we noted two significant gaps. 
Three provinces are not represented in the 
system at all, and there is little coverage 
of injuries that occur in rural areas. For 
example, there is no coverage of farm 
injuries that occur in Saskatchewan. 


14.71 Such incomplete data hamper 
CHIRPP’s ability to generate an accurate 
national picture of where and to whom 
injuries are occurring, and the types of 
injuries involved. Without this kind of 
information, it is difficult for CHIRPP to 
fully achieve its stated objective of 
reducing the number and severity of 
injuries to children. 


14.72 Health Canada should ensure 
that its Canadian Hospitals Injury 


Reporting and Prevention Program 
provides a complete national picture on 
childhood injuries from all causes. 


Department’s response: Agreed. The 
Laboratory Centre for Disease Control's 
present review of data sources for all 
surveillance systems and its subsequent 
evaluation of the efficiency and 
effectiveness of its surveillance activities 
will provide the information needed to 
enhance all of its programs. 


Weaknesses in data collection lead to 
delays in dissemination of surveillance 
information 


14.73 Dissemination of information 
represents the final stage of Health 
Canada’s health surveillance activities. 
This is the point at which information is 
placed in the hands of those who need to 
have it — the organizations responsible 
for implementing plans and strategies to 
deal with public health problems. The 
effectiveness or value of Health Canada’s 
health surveillance systems depends on 
the extent to which the information they 
produce is disseminated and used by other 
organizations and levels of government to 
improve the health of Canadians. 


14.74 Each bureau of LCDC responds 
to specific requests for surveillance 
information and distributes data through 
various publications. For example, the 
Cancer Bureau prepares detailed 
summaries of cancer incidence in specific 
areas in response to external requests, 
produces various cancer updates and 
publishes articles in peer-reviewed 
journals. As well, LCDC has made an 
effort to disseminate its surveillance 
information more effectively. In 1996 its 
Document Dissemination Division was 
created with a mandate to strengthen 
dissemination activities. Its 
responsibilities include co-ordination of 
publications, editorial production, and 
electronic dissemination. Key publications 
include the Canada Communicable 
Disease Report, published twice monthly, 
and Chronic Diseases in Canada, 
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published quarterly. As well, LCDC posts 
information on the Internet. 


14.75 However, the quality of this 
information depends on how well the 
other components of the system are 
working. Unless the “up front” 
weaknesses in data collection are dealt 
with, the availability of up-to-date and 
good-quality health surveillance 
information will be compromised. The 
problems of out-of-date and incomplete 
data have led to delays in publishing 
certain reports. 


14.76 We noted that surveillance 
information on AIDS for the period up to 
31 December 1998 was published within a 
reasonable time (in April 1999). However, 
the latest available information on 
tuberculosis and many other 
communicable diseases is for the period 
ended 31 December 1996, because Health 
Canada believes that more recent data are 
incomplete. The lack of up-to-date 
information on tuberculosis impairs 
Canada’s ability to not only monitor this 
global disease but also evaluate the 
effectiveness of the criteria used and 
process in screening for and controlling 
the entry of tuberculosis into this country. 


14.77 Similarly, while Health Canada 
has published some information on 
childhood cancer, it is based mostly on 
data from sources other than the 
Childhood Cancer Treatment and 
Outcome Surveillance System. Although 
this surveillance system has been 
collecting data since 1995, the first 
comprehensive report providing 
information on cases diagnosed in 1995 
and 1996 is not expected until the fall of 
1999, 


14.78 While individual bureaus 
periodically solicit feedback on the value 
of their dissemination activities (through 
consensus conferences, for example), 
LCDC has not formally evaluated the 
effectiveness of its dissemination 
activities — who uses the information, 
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how they use it, and whether they find it 
timely. Such evaluations are necessary to 
ensure that efforts to disseminate 
surveillance information are directed 
appropriately and that other organizations 
and levels of government are using the 
information to improve the health of 
Canadians. 


14.79 Health Canada should ensure 
the timely dissemination of surveillance 
information to those who need to have 
it. The Department should identify who 
uses the information, how they use it, 
and whether they find it timely. 


Department’s response: Agreed; 
implementation is under way. One of the 
goals in developing the National Health 
Surveillance Network is to improve the 
sharing and exchange of health 
information, and the analysis and 
dissemination of that information. As 
indicated above, this initiative is well 
advanced. 


In addition, the Laboratory Centre for 
Disease Control intends to expand the 
“Disease Surveillance On-Line” database 
in the next fiscal year to include data on 
additional diseases. This will allow data 
obtained from the provinces and 
territories, Statistics Canada or other 
sources to be posted much earlier than 
information is made available in print. 


A readership survey of Chronic Diseases 
in Canada will be conducted in the fall of 
1999 to update information obtained from 
a 1992 client survey on the readership’s 
profile, and to ascertain preferences 
regarding dissemination tools (print or 
Web) and the journal’s content. 


The Laboratory Centre 
for Disease Control 
has not formally 
evaluated the 


effectiveness of its 
A detailed, formal evaluation of the Web 


site is planted for FY.2000-2001.:The information 
Internet-based “Canadian Health dissemination 
Network”, to be implemented in 2000, will activities. 


provide the public with enhanced access 
to quality information about health 
promotion and disease prevention to 


facilitate better personal decisions about 


health. 
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Progress in Filling Surveillance 
Gaps 


14.80 In 1995, as part of a commitment 
by Health Canada to strengthen its 
surveillance capacity, LCDC launched a 
public health intelligence reinvestment 
initiative. It was designed to enable LCDC 
to expand its surveillance networks in 
order to address certain gaps in the 
surveillance of diseases such as diabetes, 
cardio-respiratory diseases, tuberculosis, 
cancer, sexually transmitted diseases and 
perinatal diseases. We examined LCDC’s 
progress in establishing surveillance 
systems in two of these areas, diabetes and 
heart disease and stroke. 


Development of a national diabetes 
surveillance system has been slow 


14.81 Diabetes is a major contributor to 
the burden of illness and health care costs 
in Canada. However, only limited national 
data on this disease are available. A 
national diabetes surveillance system 
would improve the situation, but its 
development has been slow. 


14.82 As early as 1986, the National 
Diabetes Task Force identified the 
collection of reliable data on diabetes as a 
priority. In 1995, the Diabetes Division 
was formed within LCDC. The concept of 
a national surveillance system on diabetes 
was put forward in 1996. A risk 
assessment prepared in March 1998 
identified significant gaps in surveillance 
information on diabetes, and suggested 
action to take. 


14.83 It is important that LCDC begin 
work in this area, because existing data 
sources have significant limitations. For 
example, the National Population Health 
Survey and the Aboriginal Peoples Survey 
do not address this issue to the extent 
necessary and are limited by self-reporting 
and by sample size. As a result, reliable 
information on the extent of diabetes and 
its burden on health care is not available 


to assist in the development of prevention 
and control measures. 


14.84 The 1999 federal Budget 
provided $55 million to develop a 
Canadian diabetes prevention and control 
strategy over the next three years. We 
noted that the Diabetes Division has 
begun to work with Statistics Canada, the 
Diabetes Council of Canada, the Canadian 
Diabetes Association, the Canadian 
Institute for Health Information and 
provincial/territorial governments to 
develop a national diabetes surveillance 
system. This system, which is not 
expected to be operational until 2000, 
represents another positive step toward 
closing the gap in diabetes surveillance. 


A national surveillance system has not 
yet been developed for heart disease and 
stroke 


14.85 Heart disease and stroke remain 
the major causes of death, disability and 
illness in Canada. The Department 
recognizes that a health problem of the 
magnitude and complexity of heart 
disease and stroke needs a strong 
information base to determine how the 
individual, the family, the community and 
government can best respond effectively 
and efficiently. The core functions of 
LCDC’s Cardiovascular Disease Division, 
created in September 1995, include 
national heart disease and stroke 
surveillance, risk assessments of factors 
associated with heart disease and stroke, 
and national prevention and control 
activities. 


14.86 LCDC has identified the need for 
up-to-date statistics on the number of 
people who suffer from heart disease and 
stroke and the deaths attributable to it. 
Such statistics are central to determining 
whether prevention and control efforts are 
working effectively. However, there is a 
time lag of approximately two to three 
years in the data, due mainly to data 
collection and refinement issues that need 
to be dealt with by the provinces and 
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territories, Statistics Canada and Health 
Canada. 


14.87. There is also a lack of 
good-quality national surveillance 
information on health outcomes related to 
heart disease and stroke, the risks 
influencing heart disease and stroke, and 
efforts at prevention, treatment and 
rehabilitation. LCDC has only recently 
begun surveillance work on behavioural 
risk factors. 


14.88 Health Canada is collaborating 
with other organizations, including the 
Canadian Institute for Health Information, 
Statistics Canada and non-government 
organizations, to develop an integrated 
surveillance system for heart disease and 
stroke. However, this is not expected to be 
fully operational until 2003. 


14.89 Health Canada should, in 
collaboration with other organizations, 
take steps to fill surveillance gaps by 
ensuring the timely development of 
national surveillance systems on chronic 
diseases such as diabetes, and heart 
disease and stroke. 


Department’s response: Agreed; 
implementation is under way. For several 
years, the Laboratory Centre for Disease 
Control has been working with 
stakeholders to develop national 
comprehensive chronic disease 
surveillance systems. The National 
Diabetes Surveillance System (NDSS), the 
most advanced initiative, will be 
operational by 2000 and will become a 
model for the surveillance of other chronic 
diseases. 


Current Efforts to Improve Health 
Surveillance 


14.90 Change initiatives are not new at 
the Health Protection Branch (HPB). We 
reported in 1995 (Chapter 4) and in our 
1998 follow-up (Chapter 28) that the 
Branch had undertaken change initiatives 
since 1993. We expected that the Branch 
would have taken appropriate action to 
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find solutions to identified problems in 
surveillance and related areas. We found 
that many efforts to effect change, 
including the recent transition initiative, 
have been directed to the right areas. 


14.91 The Health Protection Branch 
launched a three-year process of transition 
in August 1997. The purpose of the 
transition initiative is to help strengthen 
the Branch and its ability to better manage 
risks to the health of Canadians into the 
next century. The transition comprises five 
major components: legislative renewal, 
surveillance, science core, risk 
management framework and program 
development. 


Work has begun on strengthening 
Health Protection Branch’s future 
surveillance capacity 


14.92 An important step toward 
strengthening and expanding the Branch’s 
surveillance capacity is its support of an 
integrated national surveillance network 
for public health information. Such a 
network would integrate information from 
sources ranging from local to global. As 
part of the surveillance transition, an 
Office of National Health Surveillance has 
been created within the Branch to provide 
secretariat support for federal/provincial/ 
territorial working groups. These include 
the National Health Surveillance Network 
Working Group and the Surveillance 
Integration Design Team. The objective of 
the Office and the working groups is to 
create an integrated national health 
surveillance network that will enable 
Canada to produce accurate and timely 
information for all levels of the public 
health system. The national health 
surveillance network is intended to 
provide an infrastructure for surveillance, 
but not content. 


14.93 Extensive consultations have 
been carried out with various 
stakeholders. In the fall of 1998, at the 
request of the federal/provincial/territorial 
deputy ministers of health, the 
Surveillance Integration Design Team 
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Early concrete steps 
have been taken 
toward creating a 
national health 


surveillance network. 


released a discussion paper on an 
integrated national health surveillance 
network for Canada. Consultations on the 
discussion paper were then held with 
provincial and territorial officials and 
non-government stakeholders across 
Canada, culminating in a national 
conference on the proposed national 
framework in March 1999. 


14.94 The results of these deliberations 
were released in April 1999 as a proposal 
to develop an integrated national health 
surveillance network for Canada. The 
proposal suggests roles that each partner 
in the health surveillance network can and 
should play. It also outlines some of the 
key policies and co-ordinating 
mechanisms that are needed to provide a 
solid foundation for building an integrated 
network to meet the needs of all of the 
players while better serving the health of 
Canadians. 


14.95 Based on the proposal, the 
Department prepared a report and 
recommendations in June 1999 for 
consideration by federal, provincial and 
territorial deputy ministers of health. The 
proposal represents a significant step 
forward. An integrated network would do 
much to address a number of weaknesses 
that we have discussed in this chapter. For 
example, it would promote more 
complete, up-to-date and uniform data 
from which to produce health surveillance 
information on a number of diseases and 
injuries. We commend these efforts to 
improve national health surveillance. We 
also recognize that developing the 
network will present challenges to Health 
Canada in getting all players to work 
together. It is unclear when such an 
integrated national health surveillance 
network would be implemented, as there 
are no planned timeframes or action plans 
for implementation. It is paramount that 
all parties involved in this network remain 
committed to its implementation. 


Pilot projects are under way to improve 
access to data 


14.96 We noted that early concrete 
steps have been taken toward creating a 
national health surveillance network. A 
series of pilot projects is being carried out 
to improve access to existing databases 
and the linkages among them. The 
projects are also intended to provide tools 
for analyzing and presenting information, 
and to speed up access to the information. 
For example, the Spatial Public Health 
Information Exchange (SPHINX) 
demonstration pilot in Alberta is testing 
procedures for accessing information that 
already exists in digitized health-related 
databases. Improving access to this 
information will help to strengthen 
national public health surveillance. 


Co-operation is essential to improving 
health surveillance 


14.97 As we have noted, Health Canada 
shares responsibilities for surveillance 
with other jurisdictions. We recognize that 
this complicates its efforts to make the 
needed improvements in health 
surveillance. However, it is imperative 
that the Department impress upon the 
provinces and territories that improving 
surveillance is an urgent task, and it is in 
everyone’s interest to resolve as quickly as 
possible the problems that are standing in 
the way of “getting on with the job”. 


Performance Measurement and 
Reporting of Results 


14.98 There are several tools at 
LCDC’s disposal to facilitate better 
management of surveillance activities. 
These include evaluation, performance 
measurement and risk assessment. Unlike 
other aspects of surveillance (which, to a 
large extent, rely on the work of outside 
organizations), applying these tools is 
within LCDC’s control. 


14.99 Taken together and properly 
used, these tools provide distinct benefits 
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for an organization’s management. They 
tell management what has worked to 
produce desired results and what has not; 
whether individual systems are 
cost-effective; and, in the case of LCDC, 
whether the systems and the resources 
required to operate them are focussed on 
the areas that pose the greatest risks to 
Canadians’ health. In addition, these tools 
yield information that allows an 
organization to report publicly on the 
effectiveness of its health surveillance 
work. 


14.100 We expected that Health Canada 
would evaluate its surveillance systems to 
measure their effectiveness, including 
whether they are achieving their stated 
objectives. We also expected that it would 
have performance measures in place to 
help managers gauge the ongoing 
performance of surveillance systems, and 
that outcome measures as well as outputs 
would be emphasized. We expected that 
the Department would use evidence-based 
risk assessments to ensure that it 
concentrates its resources on the diseases 
and injuries that pose the greatest risk to 
public health. Finally, we expected public 
reporting on the effectiveness of 
surveillance activities. 


Few systems have been evaluated and 
there are no plans to evaluate others 


14.101 LCDC has no formal plans to 
evaluate its surveillance systems. No 
resources have been identified to complete 
evaluations and few have been done. 


14.102 Only two of the seven established 
systems we examined had been evaluated 
— the Canadian Hospitals Injury 
Reporting and Prevention Program 
(CHIRPP) and the National Enhanced 
Cancer Surveillance System. The latter 
system was included in an evaluation of 
the Action Plan on Health and the 
Environment. 


14.103 An external committee evaluated 
CHIRPP in 1991, two years after the 
program had begun. The review 
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recommended that both the system’s 
representativeness and the quality of 
information on the cases be improved. 
The program has not been evaluated since. 
Therefore, it is not apparent whether 
LCDC has acted on the recommendations, 
whether the program has improved, or 
whether any new problems have arisen 
over the past seven years. Our audit, 
however, noted a number of weaknesses in 
CHIRPP (see paragraphs 14.69 tol4.71) 
that suggest that Health Canada has not 
yet resolved all the problems identified in 
the 1991 review. 


14.104 The 1997 evaluation of the 
Action Plan on Health and the 
Environment assessed LCDC’s progress in 
establishing a national enhanced cancer 
surveillance system with a focus on 
environmental risks. The evaluation found 
that LCDC had made progress in 
constructing various components of the 
surveillance system, but that various tasks, 
such as initial data gathering, had not been 
completed. 


14.105 In 1993, external consultants 
completed an evaluation of LCDC’s 
activities. The evaluation focussed on 
issues such as LCDC’s role and the 
relevance and quality of the expertise and 
services it provides to clients and partners. 
It found that most of LCDC’s clients felt 
that its services and products were 
essential, but that there were delays in 
obtaining them. The findings indicated a 
lack of sufficient back-up expertise at 
LCDC during emergencies, and poor 
communication between LCDC and its 
clients. Significant changes have taken 
place at LCDC over the last six years, but 
no subsequent evaluation has been 
planned to determine whether and to what 
extent the weaknesses have been 
addressed. 


Need to develop appropriate indicators 
for measuring performance 


14.106 Whereas evaluation is review that 
is carried out periodically, performance 
measurement involves the ongoing 
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measurement of inputs, outputs, and 
outcomes. Performance measurement 
helps managers know whether a 
surveillance system is on track. Central to 
measuring performance are performance 
indicators, which specify the terms in 
which performance or success will be 
measured. Performance indicators may be 
output-focussed, such as the number of 
seminars held or brochures distributed. Or 
they may be outcome-focussed, such as 
surveillance information used in program 
or policy decisions that lead to the desired 
reduction in the occurrence of diseases or 
injuries. 


14.107 LCDC has developed objectives 
for the surveillance systems that we 
reviewed. It has begun to develop a 
framework for better measurement of its 
performance, but it is not complete. 
LCDC has not yet finished developing 
indicators to measure performance that 
focus on outcomes as well as inputs. The 
performance indicators it has developed 
tend to be process- or output-oriented. For 
example, the Cancer Bureau lists items 
such as “client satisfaction” and 
“publication in scientific journals and 
other publications” as performance 
indicators of its childhood cancer 
program. As well, few indicators focus on 
specific surveillance systems. 


14.108 Until LCDC finishes developing 
its performance measurement framework 
and includes in it outcome-oriented 
performance indicators, it will have some 
difficulty knowing what works and what 
does not, and when to take corrective 
action. 


Only limited use of risk assessment 


14.109 Risk assessment is an important 
tool that can be used to direct surveillance 
effort to best advantage — that is, 
commensurate with the risk to public 
health that a disease represents. Risk 
assessment techniques can be used, for 
example, to determine the risks that one 
disease poses to public health compared 


14-26 


with another. By using the results of risk 
assessment, management at LCDC can 
focus surveillance activities and resources 
more sharply on the diseases and injuries 
that pose the highest risk. In essence, then, 
risk assessment can be used to help ensure 
that effort is spent where it will have the 
most impact. 


14.110 We noted that the Advisory 
Committee on Epidemiology, working 
jointly with LCDC, uses a risk-based 
approach to select and prioritize 
communicable diseases that are to be 
reported nationally. As well, in 1995, after 
consultations and using criteria to rank the 
severity of surveillance gaps, the 
Department launched its public health 
intelligence reinvestment initiative. This 
included a greater focus on chronic 
disease surveillance. 


14.111 However, in the specific 
surveillance systems we examined, we 
found few cases in which a thorough risk 
assessment of individual diseases (as a 
prelude to surveillance) had been done to 
assess their risks in comparison with 
others and to outline options for action. 
An exception was the health risk report 
and recommended control strategy 
completed for diabetes, which included an 
epidemiological assessment, analyses of 
knowledge gaps and jurisdictional 
considerations, and a control strategy. 


14.112 Instead, most diseases considered 
to be candidates for surveillance are 
identified through formal and informal 
consultations rather than objective, 
evidence-based risk assessments. Through 
these consultations, the Department has 
identified major gaps in surveillance 
coverage (for example, chronic diseases 
such as musculoskeletal diseases, nervous 
system and mental disorders and skin and 
related diseases). 


14.113 The Health Protection Branch has 
taken initial steps to improve the way it 
manages risk. As part of its transition 
initiative, the Branch has recognized the 
need to consider not only the risks 
associated with individual diseases but 
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also their relative importance. As well, the 
Branch has recognized the need to 
improve the way it communicates risk 
issues to its clients and other stakeholders. 


Need for improved reporting of results 
to Parliament 


14.114 LCDC reports to Parliament only 
limited information on the effectiveness of 
its surveillance systems and activities. We 
reviewed departmental Estimates/Plans 
and Priorities documents and Performance 
Reports for the last five years and found 
few references to either the rationale for 
the surveillance systems or their 
objectives and effectiveness. As well, the 
results of the 1993 LCDC evaluation were 
not reported to Parliament. 


14.115 Health Canada should 
strengthen the evaluation, performance 
measurement and reporting of results of 
its health surveillance systems and 
activities. 


Department’s response: Agreed. Steps are 
being taken now to implement this 
recommendation. 


Conclusion 


14.116 The success of national health 
surveillance depends largely on the ability 
of federal, provincial, territorial and local 
health departments and organizations to 
develop partnerships, to collaborate and to 
understand clearly what needs to be done 
to prevent and manage disease outbreaks, 
who should do it, and when and how it 
should be done. 


14.117 Health Canada has no formal 
arrangements with the provinces and 
territories covering the common standards 
and procedures for collecting and 
exchanging data on communicable 
diseases. In addition, there are few formal 
agreements or protocols for dealing with 
the entry into Canada of serious 
communicable diseases, disease outbreaks 
and threats to public health. 
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14.118 We found weaknesses in the key 
surveillance systems that we looked at. 
The systems were not enabling Health 
Canada to effectively monitor 
communicable diseases such as influenza, 
AIDS, tuberculosis and food-borne 
diseases. There are significant gaps in 
surveillance information on certain 
chronic diseases such as cancer, diabetes, 
and heart disease and stroke. National 
surveillance information on childhood 
injuries does not yield a complete picture. 
In some cases, weaknesses in data 
collection lead to delays in the 
dissemination of surveillance information. 


14.119 Health Canada needs to 
strengthen its procedures to measure 
performance and report results. It also 
needs to make better use of risk 
assessment in its health surveillance 
activities. 


14.120 Work has begun on strengthening 
the Health Protection Branch’s future 
surveillance capacity, including the 
proposal for an integrated national health 
surveillance network. Such a network 
would do much to address some of the 
weaknesses that we observed. 


14.121 We recognize that LCDC is not 
solely responsible for a number of these 
weaknesses. Nonetheless, the weaknesses 
need to be fixed, and LCDC needs to 
champion change by ensuring that all 
players work together to improve the 
surveillance on which Canadians rely to 
protect their health. 


Department’s comments: The federal 
government has launched an ambitious 
initiative to develop a National Health 
Surveillance Network. This is a complex 
undertaking that entails a significant 
collaboration between Health Canada and 
its provincial and territorial partners. The 


full impact of the initiative will not be 


realized until the entire Network is 
operational, but improvements are being 
made annually as components of the 
network are put in place. 
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ey About the Audit 


Objectives 
The objectives of the audit were to determine whether: 


e there are common standards and protocols to facilitate collaboration and sharing of information among 
all players in the health surveillance process; 


e required surveillance systems are in place to provide timely and relevant information to anticipate, 
prevent and respond to health threats and emerging health risks; 


e procedures to measure performance and report results are adequate; 
e there is a rational basis for determining where and how health surveillance should be undertaken; and 


e progress has been made in establishing solutions to identified problems. 


Scope 


The audit examined Health Canada’s surveillance activities relating to diseases, both communicable and 
non-communicable, and associated behaviour risk factors, as well as injuries. We examined surveillance 
systems designed to collect, analyze and disseminate information in order to anticipate, prevent and respond 
to health threats and emerging health risks. This included a detailed review of 10 of the approximately 

40 functioning surveillance systems the Department maintains in the areas of communicable and 
non-communicable diseases and injuries. Taken together, the surveillance systems we reviewed covered 
diseases and injuries that pose significant threats to health and represent a significant economic burden to 
society. In addition, we examined Health Protection Branch transition initiatives relating to surveillance, risk 
management and legislative renewal. 


We had extensive discussions with departmental staff, selected provincial and local health officials, other 
agencies and selected foreign national institutions involved in health surveillance activities. As well, we had 
discussions with representatives of the Canadian Food Inspection Agency. We reviewed various studies and 
reports published by Health Canada, provincial and local health agencies and other national health bodies, 
and other documentation. 


The audit did not examine the management of risks to health relating to consumer goods, medical devices and 
therapeutic products. The audit did not cover diseases relating to animals. 


The quantitative information in this chapter that has been drawn from government and non-government 
sources or departmental databases has been checked for reasonableness but has not been audited. 


Criteria 
We expected that: 


e all players in health surveillance would clearly understand Health Canada’s role and responsibilities in 
the national health surveillance process; 


e a full range of documented protocols and procedures would exist that indicate clearly what each player 
should do when a disease outbreak or health threat occurs; 


14-28 Report of the Auditor General of Canada — September 1999 


National Health Surveillance: Diseases and Injuries 


e all players in the health surveillance process would use common standards and protocols for classifying, 
collecting and reporting data on diseases and injuries to ensure that all information is comparable across 
the country; 


e Health Canada’s surveillance systems for monitoring diseases and injuries would enable it to collect, 
analyze and disseminate all information necessary to help anticipate, prevent and respond to existing and 
emerging health risks; 


e there would be procedures to measure the effectiveness of health surveillance activities and report results; 


e Health Canada’s health surveillance activities would be based on a sound risk-benefit approach and a 
rational priority setting framework; and 


e@ appropriate action would be taken by Health Canada in establishing solutions to identified problems. 


Audit Team 


Assistant Auditor General: Maria Barrados 
Principal: Ronnie Campbell 
Director: Gerry Chu 


Julie Charron 
Anil Risbud 
Glenn Wheeler 


For information, please contact Ronnie Campbell. 
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Chapter 15 


Management of a Food-Borne 
Disease Outbreak 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 


Accountants. The numbered paragraphs in bold face represent recommendations. 


Table of Contents 


Main Points 


Introduction 


Many players are involved in the investigation and control of food-borne disease outbreaks 
Many young children were affected in a nation-wide outbreak of food-borne disease 

Focus of the audit 

Completeness of relevant information 


Observations 


Lack of timely exchange of information to identify scope of the outbreak 

LCDC was not well prepared to manage a disease outbreak 

Quick identification of contaminated food product 

CFIA did not share certain distribution information with provincial public health officials 
Further recalls were announced 

Several plants were inspected in an effort to determine the source of contamination 
Disagreement on the need to reinspect a plant 

Lack of formal protocols and established procedures to investigate outbreak 

Lack of transparency in post-outbreak reviews 


Conclusion and Recommendations 
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Management of a Food-Borne 
Disease Outbreak 


Main Points 


15.1 In the spring of 1998, there was a nation-wide outbreak of a food-borne disease; it was one of the largest 
outbreaks of food-borne disease in Canadian history and involved the investigation of more than 800 reported 
cases across Canada. Over 80 percent of the affected were children under 15 years of age. At least 60 were 
hospitalized. 


15.2 Some important aspects of the response to the outbreak worked well, but others did not. The 
contaminated product was identified quickly and its removal from points of sale was started immediately after the 
issuing of a recall. However, there was a lack of timely exchange of information to identify the scope of the 
outbreak. There was also a lack of full co-operation among the agencies involved in the response to this outbreak. 
The Canadian Food Inspection Agency (CFIA) did not share certain distribution information when requested by 
provincial public health officials to assist in the investigation. In addition, the CFIA’s abrupt decision not to lead 
one of the plant inspections resulted in confusion and unnecessary delay. 


15.3 A formal framework is needed that sets out clearly the roles and responsibilities of Health Canada’s 
Laboratory Centre for Disease Control (LCDC) in relation to those of other participants, in order to guide the 
response to threats to public health. Many individuals could have avoided this illness had the federal and 
provincial health departments acted more quickly. 


Background and other observations 


15.4 The case described in this chapter illustrates many of the issues discussed in Chapter 14 on National 
Health Surveillance and the management of outbreaks and threats to public health. The audit looked at how 
federal and provincial agencies — Health Canada’s Laboratory Centre for Disease Control and Food Directorate, 
the Canadian Food Inspection Agency, and provincial and local public health departments — responded to this 
nation-wide outbreak of a food-borne disease. 


iss Food-borne diseases have important implications because of the wide distribution of food products and 
the resulting potential for affecting very large numbers of people spread over wide geographic areas. 


15.6 The activities of the Laboratory Centre for Disease Control involve the timely investigation and control 
of disease outbreaks, often in collaboration with provinces and other federal agencies — in particular, with the 
Canadian Food Inspection Agency in the event of a disease caused by food. The CFIA is responsible for 
enforcement actions in food-related emergencies and is to take a lead role in investigations and co-ordination of 
food safety emergency responses. Health Canada’s Food Directorate is responsible for assessing the effectiveness 
of the CFIA’s food safety activities. Provincial and local medical officers of health have a legislative mandate to 
investigate disease outbreaks, and provincial laboratories provide laboratory services. 


1537 We found that LCDC was not well prepared to manage disease outbreaks. It had no established operating 
procedures to respond to food-borne disease outbreaks. In addition, there were no formal protocols between 
Health Canada, the CFIA and the provinces that clearly defined procedures and the role of LCDC in relation to the 
roles of other participants in the investigation of disease outbreaks. 
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15.8 There was a lack of transparency in the post-outbreak reviews undertaken by LCDC and by the CFIA. In 
such situations, particularly given that the cause of the contamination was never found, we believe it is in the 
interest of public health that all participants contribute to, and learn from, such reviews. 


Responses to our recommendations from Health Canada and the Canadian Food Inspection Agency are 
included in this chapter. Both Health Canada and the Canadian Food Inspection Agency concur with the 
recommendations and have agreed to take corrective action. In some cases, this action is already under way. 
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Introduction 


15.9 This chapter flows from our audit 
of National Health Surveillance, including 
the management of outbreaks and threats 
to public health (reported in Chapter 14). 
In this chapter, we examine the 
management of a recent outbreak of a 
food-borne disease. The scope of the 
examination extends beyond health 
surveillance to address the way Health 
Canada’s Laboratory Centre for Disease 
Control (LCDC) and the Canadian Food 
Inspection Agency (CFIA) interacted with 
other participants in responding to the 
outbreak. 


15.10 This case was one of the largest 
food-borne disease outbreaks in Canadian 
history. It involved many provinces and 
agencies. Some aspects of the response to 
the outbreak worked well, particularly the 
quick identification of the contaminated 
product. However, other aspects did not 
work as well. The case illustrates many of 
the issues discussed in the chapter on 
National Health Surveillance, including 
the need for timely sharing of information, 
better collaboration and co-operation 
among federal and provincial agencies, 
and a formal framework that would 
clearly specify LCDC’s role in relation to 
the roles of other participants in order to 
guide the response to threats to public 
health. 


Many players are involved in the 
investigation and control of food-borne 
disease outbreaks 


15.11 The core activities of the 
Laboratory Centre for Disease Control 
within Health Canada’s Health Protection 
Branch (HPB) are national health 
surveillance, disease prevention and 
control. These activities involve the 
timely investigation and control of disease 
outbreaks, often in collaboration with 
provinces and other federal agencies — in 
particular, with the Canadian Food 
Inspection Agency in the event of a 
disease caused by food. 
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15.12 Created in 1997, the CFIA is 
responsible for enforcing the Food and 
Drugs Act and other legislation covering 
all foods sold in Canada (foods as defined 
in the Canadian Food Inspection Agency 
Act). It is responsible for enforcement 
actions in food-related emergencies and 
has a lead role in co-ordination of 
emergency response and investigation of 
food-borne illness outbreaks. 


15.13. Health Canada, through its Food 
Directorate of the Health Protection 
Branch, establishes policies and standards 
for the safety and nutritional quality of 
food sold in Canada. It is also responsible 
for assessing the effectiveness of the 
CFIA’s food safety activities. 


15.14 — Provincial and local medical 
officers of health have a legislative 
mandate to investigate disease outbreaks, 
and provincial laboratories provide 
laboratory services. 


Many young children were affected in a 
nation-wide outbreak of food-borne 
disease 


15.15 Food-borne diseases have 
important implications because of the 
wide distribution of food products and the 
resulting potential for affecting very large 
numbers of people spread over wide 
geographic areas. Resolving an outbreak 
requires the timely identification of the 
disease, finding the cause of the outbreak, 
recalling and removing the food product, 
treating the infected, and preventing a 
recurrence. All the players thus need to 
work together, share information, act 
quickly to protect the health of the public 
and regard safety as a first priority. 


15.16 A nation-wide outbreak of a 
food-borne disease occurred in March and 
April 1998; it involved the investigation 
of more than 800 reported cases across 
Canada. Most of those affected were 
young children. LCDC indicated that 
likely many more were affected, possibly 
10 times more. A number suffered severe 
gastroenteritis, and at least 60 were 
hospitalized. It was one of the largest 


A 1998 nation-wide 
outbreak of a 
food-borne disease 
involved the 
investigation of more 
than 800 reported 
cases across Canada. 
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No protocols or 
guidelines existed that 
would specify when or 
under what 
circumstances to 
advise the Laboratory 
Centre for Disease 
Control (LCDC) of an 
outbreak. 


food-borne disease outbreaks in Canadian 
history. 


Focus of the audit 


15.17 We examined the way Health 
Canada and the Canadian Food Inspection 
Agency managed this outbreak in 
collaboration with other jurisdictions. The 
audit focussed on the identification of the 
outbreak and the communication of its 
occurrence, the investigation to determine 
the source and cause of the outbreak, the 
recall and removal of the contaminated 
food product, and post-outbreak review 
procedures. Further details on the audit 
scope, objectives and criteria are 
presented at the end of the chapter in 
About the Audit. 


Completeness of relevant information 


15.18 This audit required that we obtain 
information from a variety of 
organizations. Health Canada was fully 
co-operative and provided all of the 
information that we requested. The 
provincial and local organizations were 
also very co-operative in responding to 
our requests. 


15.19 Throughout much of the audit 
process, the Canadian Food Inspection 
Agency displayed a lack of willingness to 
provide us, on a timely basis, with the 
information that we required to do our 
work. However, CFIA officials did 
become more responsive in their efforts to 
supply us with information toward the end 
of the audit process, and confirmed that 
they had provided all available relevant 
information. In addition, CFIA officials 
advised us that they had not kept records 
of all key meetings and decisions held in 
the course of their investigation and that 
other, potentially relevant information had 
been lost prior to the commencement of 
the audit. Given these circumstances, we 
do not have our usual degree of assurance 
with regard to the completeness of the 
record of events. 


Observations 


Lack of timely exchange of information 
to identify scope of the outbreak 


15.20 The Laboratory Centre for 
Disease Control relies largely on the 
provinces and territories to report cases of 
communicable diseases. The data it 
receives include laboratory-confirmed 
cases, which are normally reported to it 
weekly by the provinces and territories. In 
addition, provincial laboratories often 
send specimens to national reference 
laboratories at LCDC to have the type of 
pathogen confirmed. 


15.21. In March 1998, the Province of 
Ontario’s Central Public Health 
Laboratory in Toronto noted a substantial 
increase in the number of confirmed cases 
of Salmonella Enteritidis. From 6 March 
to 18 March 1998, 42 cases were 
confirmed, compared with an average of 
51 confirmed cases in the entire month of 
March each year from 1993 to 1997. The 
cases were believed to be widely 
distributed throughout Ontario. 


15.22. On 18 March, the Central Public 
Health Laboratory notified the Public 
Health Branch of the Ontario Ministry of 
Health about the 42 cases. Given the 
information it had on 18 March, the Public 
Health Branch was in a position to inform 
LCDC that an outbreak was likely under 
way. Ontario Public Health Branch 
officials recognized this but elected to 
conduct further analysis. Their analysis 
showed that in the five- to nine-year-old 
age group, the number of cases was five 
times higher than normal. The results of 
this analysis were not shared with LCDC 
until 26 March, when LCDC called the 
first teleconference. No protocols or 
guidelines existed that would specify 
when or under what circumstances to 
advise LCDC of an outbreak. 


15.23. The confirmed cases were 
reported to LCDC only through the 
routine surveillance system, which did not 
have the capacity to provide an early 
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signal of a significant increase in the 
number of cases. 


15.24. While Ontario was experiencing 
significantly more cases of Salmonella 
Enteritidis, similar cases started to show 
up in several other provinces. On 20 
March, three Newfoundland children with 
severe gastroenteritis were seen in a 

St. John’s hospital and specimens were 
sent for examination for Salmonella 
organisms. On the same day, the 
Newfoundland Public Health Laboratory 
confirmed a similar case in Gander. The 
provincial laboratory notified the 
provincial disease control office to alert 
other regions in the province to a possible 
outbreak. 


15.25. On 23 March, six more suspected 
cases of Salmonella Enteritidis were 
identified, mostly children in three 
Newfoundland communities. 
Newfoundland provincial public health 
officials believed a province-wide 
outbreak was occurring. The chief medical 
officer of Newfoundland placed a call on 
25 March to LCDC in Ottawa. The 
Newfoundland provincial laboratory also 
sent specimens to LCDC’s laboratory for 
confirmation of the type of Salmonella. 


15.26 Given the information it had on 
18 March, Ontario was in a position to 
suspect an outbreak and alert LCDC. 
Ontario’s failure to do so resulted in a 
delay of a full week in the start of the 
investigation into the outbreak. 


LCDC was not well prepared to manage 
a disease outbreak 


15.27. The phone call to LCDC by the 
chief medical officer of Newfoundland did 
not produce a quick, smooth exchange of 
information. She did not know whom to 
call at LCDC. When contacted, LCDC 
officials were not sure who in their 
organization was responsible. No one in 
LCDC was formally tasked with 
managing food-borne disease outbreaks. 
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15.28 After several calls, a LCDC 
epidemiologist in Guelph took 
responsibility for co-ordinating the 
outbreak investigation — on her own 
initiative, as no one at LCDC was 
formally tasked with co-ordinating action 
in outbreaks. There was also confusion in 
LCDC about who should be the 
spokesperson for Health Canada. 


Ontario’s failure to 
alert LCDC resulted in 
a delay of a full week 


15.29 We believe it is essential that all in the start of the 
participants know whom to contact at : de te heer 

LCDC when there is a disease outbreak investigation into the 
and who is responsible for managing it. outbreak. 


Not knowing can mean unnecessary 
delays in moving to protect the health of 
the public. 


Quick identification of contaminated 
food product 


15.30 Despite the problems that we 
have described, it appears that the 
individuals involved worked well together 
and moved quickly to identify the 
contaminated food product. 


15.31 On 26 March, the first 
teleconference call among officials from 
LCDC, Ontario and Newfoundland 
confirmed that there was a similar 
outbreak in Ontario, with over 114 cases. 
During the discussion, Newfoundland 
reported a total of 22 probable cases in 
that province. After learning of the 
situation in Newfoundland, Ontario sent 
isolates to LCDC’s laboratory for 
comparison of the type of Salmonella. On 
the same day, LCDC notified the CFIA in 
Ottawa of the outbreak. 


The individuals 
involved appeared to 
work well together and 
moved quickly to 


identify the 
15.32 Once the various parties (LCDC, ; 
the CFIA, Ontario and Newfoundland contaminated food 
public health officials) were brought product. 


together in a response team, the 
contaminated product was identified 
quickly. Health Canada, in collaboration 
with provincial public health officials, was 
responsible for conducting 
epidemiological investigations to 
determine the product responsible for the 
outbreak. The CFIA was responsible for 
conducting investigations to determine the 
cause of contamination. In addition, the 
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CFIA was responsible for enforcement 
actions, which included ensuring that food 
recalls were carried out by the food 
manufacturer as required and monitoring 
the effectiveness of the recalls. 


15.33, On 27 March, packaged lunch 
products were identified as a suspected 
food. On the same day, Newfoundland 
received test results from the federal 
laboratory confirming that the specimens 
belonged to the same type of Salmonella. 
The first public announcement of the 
food-borne illness was made in late 
afternoon that day by the Newfoundland 
health department. It issued a news release 
advising that there was an outbreak of 
Salmonella gastroenteritis in the province. 
The release noted that Ontario was 
experiencing a similar outbreak. 


15.34 Based on information from 
patients, provincial health officials and 
CFIA food inspectors started collecting 
suspected food samples from food outlets 
and shops in Newfoundland and Ontario. 
Testing began on 28 March in four 
laboratories. 


15.35 On 30 March, the Newfoundland 
laboratory provided a preliminary result: a 
likely positive for Salmonella in a lunch 
product. CFIA officials in Ontario relayed 
this information to the company that 
manufactured the lunch product. Company 
officials started to pull together 
distribution and production records on the 
product and shared the information with 
the CFIA. 


15.36 By 31 March, the joint 
collaborative effort was able to isolate the 
source of the food product. CFIA 
laboratories in Ontario and Nova Scotia 
confirmed that the cheese in the lunch 
product was contaminated with 
Salmonella Enteritidis. The CFIA 
considered a food recall to be clearly 
necessary. Health Canada supported the 
CFIA’s decision. 


15.37. The CFIA could have forced the 
manufacturer to recall the food product or 


allowed the manufacturer to initiate a 
voluntary recall. In this case, the 
manufacturer decided to issue a voluntary 
class I recall of its products containing the 
implicated cheese, along with a national 
health hazard alert (a class I recall 
involves a situation in which the use of a 
product may cause serious adverse health 
consequences or death). 


CFIA did not share certain distribution 
information with provincial public 
health officials 


15.38 On 31 March, the company 
issued a recall of four of its products. The 
health hazard alert advised the public not 
to consume four of its lunch products 
because one of the cheese ingredients 
might contain the bacteria responsible for 
Salmonella infection. In a voluntary 
recall, the company is responsible for 
removing the product at points of sale. 
The CFIA is responsible for conducting an 
effectiveness check of the recall process. 


15.39 The CFIA undertook recall 
effectiveness checks to ensure that retail 
stores had been informed of the recall and 
the product removed from food outlets. 
This comprised telephone calls to 
consignees and some visits to retail outlets 
by CFIA staff. Officials told us that the 
recall effectiveness check was intended to 
cover 100 percent of the stores selling the 
product. 


15.40 The Ontario Ministry of Health 
had two concerns about risks to public 
health. First, it was concerned that the 
CFIA’s recall effectiveness checks would 
not be carried out quickly enough. 
Second, it was concerned that about one 
third of the Salmonella cases being 
investigated in Ontario had no link to the 
suspected food product, and therefore an 
analysis of the distribution of the cheese 
(supplied by a different company) would 
help in the investigation. 


15.41 In an effort to address the first 
concern, the Ontario chief medical officer 
offered to have local public health officers 
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visit retail stores and help to ensure that 
the product was removed from sale 
immediately following the announcement 
of the recall. 


15.42 Ontario officials noted that the 
CFIA did not want the Ontario Ministry of 
Health to participate in this way. They 
informed us that they visited retail stores 
in spite of the CFIA’s opposition. The 
value of the provincial participation was 
evident from the fact that the inspectors 
found a number of outlets with the product 
still on the shelves. For example, 14 of 
Ontario’s 37 public health units reported 
that on 3 April the recalled product was 
still on the shelves of 134 stores. 


15.43 To deal with their second 
concern, Ontario Ministry of Health 
officials wanted to compare the 
distribution of reported Salmonella cases 
with the distribution of the cheese in 
question. Ontario officials informed us 
that they requested the distribution list of 
all the cheese made from the suspect 
source several times in late March and 
early April. However, the CFIA did not 
provide the information at that time. 
Several months later, the CFIA made the 
information available to Ontario, but by 
that time it was too late to be of assistance 
in the investigation. 


15.44 However, CFIA officials have a 
different recollection as to what occurred. 
They believe that there was a high degree 
of co-operation between CFIA staff and 
Ontario public health officials. 


Further recalls were announced 


15.45 On 9 April, after discussions with 
the CFIA, the company that manufactured 
the lunch products issued a second 
voluntary class I recall, with a public 
warning, that included additional 
products. This was due to possible 
contamination of a different type of 
cheese contained in one of its lunch 
products that had not been included in the 
earlier recall. The recall was issued 
promptly by the manufacturer. 
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15.46 Early in the afternoon of 

13 April, the CFIA laboratory identified 
Salmonella in two cheese sticks sampled 
from one of the 13 lots of cheese at the 
factory of the company that had supplied 
the cheese to the lunch product 
manufacturer. All the cheese sticks made 
from this lot were on hold in the 
warehouse of this company. The 
implicated lot had also been used in one 
of the recalled lunch products. 


15.47. On 14 April, the CFIA 
recommended that the company that had 
produced the cheese take immediate 
action to recall the product from all levels 
of trade and issue a public announcement 
to all consumers. However, the company 
disagreed with this recommendation. 


15.48 On 15 April, the company finally 
agreed to issue a class II recall of three 
brands of its cheddar cheese products as a 
precautionary measure and to make a 
public announcement. (A class II recall, 
with or without a public warning, 
normally applies to a situation in which 
the use of a product may cause temporary 
adverse health consequences or where the 
probability of serious adverse health 
consequences is remote). The company 
issued a public advisory that afternoon. 


15.49 We noted that CFIA inspectors 
were still checking food outlets until late 
April. On 29 April, a Health Canada field 
epidemiologist involved in the 
investigation found eight packages of the 
recalled products still in a Mississauga 
store. 


Several plants were inspected in an 
effort to determine the source of 
contamination 


15.50 In the course of the investigation, 
the CFIA inspected a total of eight food- 
processing plants. During these 
inspections, it performed environmental 
checks, among other things, and took 
numerous food samples for further testing 
at CFIA laboratories. Some plants were 
inspected more than once. On several 


Report of the Auditor General of Canada — September 1999 


Ontario officials 
informed us that they 
visited retail stores in 
spite of opposition by 
the Canadian Food 
Inspection Agency. 


15-11 


Management of a Food-Borne Disease Outbreak 


Concerns remained 
about the water quality 
at one of the plants 
where the cheese had 
originated. 


The Canadian Food 
Inspection Agency 
decided that it would 
no longer lead the 


inspection of the plant. 


occasions, the inspection team consisted 
of personnel from other agencies. 


15.51 After a number of inspections, 
nothing had been found that would allow a 
conclusion to be drawn as to the cause of 
the contamination. Concerns remained 
about the water quality at one of the plants 
where the cheese had originated. 


Disagreement on the need to reinspect a 
plant 


15.52 During an interagency conference 
call on 20 April, a decision was made to 
follow up at the cheese plant with an 
interagency team to be led by the CFIA. 
CFIA officials had previously inspected 
this plant on 3, 4, 6 and 14 April. The 
local public health unit also took water 
samples at this plant on 17 and 20 April. 


15.53, On 22 April, an interagency 
meeting chaired by the CFIA was held, 
and plans were drawn up for an 
interagency inspection of the plant, which 
was to take place the following day. The 
CFIA notified the company of the plan 
orally and by letter. Company officials 
contacted CFIA management and urged 
better co-ordination among the federal 
agencies involved in the ongoing 
investigation. In addition, they raised 
questions regarding the nature and 
purpose of the upcoming inspection of the 
plant, which was to be led by CFIA 
officials. However, no objections were 
raised to the reinspection or to CFIA 
involvement. The CFIA, following its 
standard practice, contacted its Minister’s 
office to advise it of the situation. 


15.54 On 22 April, the president of the 
company wrote to the CFIA, Health 
Canada and its Minister, complaining 
about a lack of co-ordination of effort and 
communication among the federal 
agencies. He alleged that his company 
was being treated unfairly, and was 
experiencing serious and preventable 
disruption as a result of the lack of 
co-ordination between the CFIA and 
Health Canada. 


15.55 The president of the company 
also called the Minister responsible for the 
CFIA and followed the call with a letter. 
CFIA officials told us that the Minister’s 
office had maintained that this file, as all 
others, would be handled according to 
normal operating procedures. 


15.56 We were told by CFIA officials 
that they responded orally to the 
company’s letters to CFIA and their 
Minister and that there was no formal 
reply. The Minister of Health later 
responded that an outbreak of this nature 
was an extremely serious matter, requiring 
an intensive and thorough investigation, 
and that there were valid reasons for 
focussing on the company’s 
manufacturing processes. 


15.57. CFIA officials told us that senior 
management decided, upon reviewing the 
file, that based on their analysis of 
previous inspections there was no further 
need for the Agency to inspect the plant. 
The Agency decided that it would no 
longer lead the inspection of the plant. In 
effect, it changed its role to one of 
assisting Health Canada in determining 
the cause of contamination, rather than 
maintaining a lead role in the 
investigation. 


15.58 Its partners, Health Canada and 
the provincial and local public health 
officials, persisted. They felt that there 
were sufficient reasons to have the plant 
reinspected. 


15.59 Ontario public health officials 
and Health Canada decided to go ahead 
with the inspection. However, leading it 
was not within Health Canada’s 
jurisdiction. In the absence of CFIA’s 
leadership, the Ontario Ministry of Health 
had to call upon the local medical officer 
of health to lead the inspection. 


15.60 One result of this confusion over 
who would lead the inspection was that it 
was delayed for one day. The plant had 
been in operation on 21—22 April. The 
company closed the plant for renovations 
beginning on 23 April. Therefore, when 


Report of the Auditor General of Canada — September 1999 


the inspection went ahead on 24 April, the 
plant was not in operation. Officials from 
the provincial and local public health 
departments, the provincial food 
inspection branch, Health Canada and the 
CFIA visited the plant. However, the two 
CFIA representatives indicated to the 
other participants that they were there as 
“resource people” and would not write or 
prepare any CFIA report on the 
inspection. One of the CFIA 
representatives advised us that she was 
told by her manager to offer technical 
support and answer questions only. 


15.61 The inspection revealed 
contamination in the plant’s water supply 
and identified several other opportunities 
for contamination of cheese. The team 
concluded that the water at the time of the 
inspection was not potable. Although the 
water was mainly used for cleaning, 
nevertheless the plant was ordered by 
local public health and CFIA officials to 
remain closed until a water disinfection 
system could be installed. However, the 
investigation could not determine the 
cause of the contamination of the cheese. 


15.62 Once the inspection was 
completed, it was the provincial and local 
officials who prepared the reports. CFIA 
representatives, however, assisted in a 
technical review of the inspection report 
and participated in the briefing of 
company officials on the results of the 
inspection. 


15.63 However, officials had differing 
recollections as to what had occurred and 
what their respective roles had been. 
While all of the other participants 
maintained that CFIA officials had not 
actively participated in the inspection, 
CFIA officials claimed that they had. 


15.64 The CFIA’s withdrawal as the 
lead for the inspection resulted in 
unnecessary delay and jurisdictional 
confusion among participants. It also 
hampered their collaboration. We believe 


Report of the Auditor General of Canada — September 1999 


Management of a Food-Borne Disease Outbreak 


it is vital that any threat to public health 
be investigated co-operatively. 


Lack of formal protocols and 
established procedures to investigate 
outbreak 


15.65 At the time of the outbreak, 
LCDC had no established operating 
procedures to respond to food-borne 
disease outbreaks. The action it took was 
largely ad hoc. A draft emergency 
response plan existed at LCDC but was 
not consulted by those directly involved in 
the investigation. 


The action that LCDC 
took was largely ad 
hoc. 


15.66 As noted in paragraph 14.38 of 
Chapter 14, several memoranda of 
understanding have been established 
between Health Canada and the CFIA 
with respect to food safety. At the time of 
the outbreak, however, there were no 
formal protocols between Health Canada 
and the CFIA and the provinces that 
clearly defined procedures and the role of 
LCDC in relation to the roles of other 
participants in the investigation of disease 
outbreaks. 


15.67 As mentioned in paragraph 14.39 
of Chapter 14, a food-borne illness 
response protocol is currently being 
developed by Health Canada, the CFIA, 
and provincial/territorial governments. 


15.68 Officials told us that they expect 
that the protocol will describe the roles 
and responsibilities of the organizations 
and provide a framework for a 
co-ordinated response to food-borne 
disease outbreaks. They anticipate that 
this will result in a more co-operative and 
rapid response to these outbreaks. 


15.69 Officials also told us that Health 
Canada and its partners recognize the 
importance of timely exchange of 
information and notification of outbreaks 
of food-borne diseases. Early concrete 
steps have been taken toward creating a 
national health surveillance network. A 
series of pilot projects is being carried out 
to improve access to existing databases 
and the linkages among them. For 
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LCDC organized a 
post-outbreak review 
meeting but the 
Canadian Food 
Inspection Agency 


declined to participate. 


example, the Canadian Integrated Public 
Health System (CIPHS) is designed to 
provide data communication links 
between laboratories and public health 
officials on a “real time” basis. Officials 
anticipate that this system can help reduce 
the time it takes to recognize and 
communicate the existence of potential 
outbreaks. 


Lack of transparency in post-outbreak 
reviews 


15.70 The poor communication and 
lack of formal arrangements indicate a 
clear need for the participants to work 
better as a team. In addition, there were 
few formal records kept of discussions and 
decisions made at interagency 
teleconferences during the investigation. 

A post-outbreak review was vital to ensure 
that lessons were learned and that such a 
situation would not happen again. 


15.71 LCDC organized a post-outbreak 
review meeting on 11 May 1998 in an 
effort to learn how the management of an 
outbreak such as this one could be 
improved. However, the CFIA declined to 
participate. It informed us that there were 
two reasons for that decision. First, it 
noted that its investigation was still 
ongoing at that time, and that a 
post-outbreak review would be premature. 
Second, it was concerned that anything its 
officials might say at such a meeting 
could be used in any potential lawsuit. 


15.72 The post-outbreak review proved 
to be of limited value. LCDC did not 
circulate the minutes of the meeting. Nor 
did it provide any of the participants with 
a summary of the lessons learned. 


15.73. The CFIA held its own 
post-investigation review on 10 
December. This was strictly an internal 
meeting, and none of the other agencies 
was invited. The goal was to examine the 
investigation and recall of the food 
product in order to improve investigations 
and emergency responses within the 
CFIA. The agenda for this meeting stated 


that the participants would discuss, among 
other things, the recall reality, chronology 
of events, investigation overview, what 
went well and areas for improvement, 
strategy development, and 
recommendations. However, CFIA 
officials told us that the meeting was not 
specifically for the purpose of discussing 
the outbreak, and there were no minutes of 
discussion relating to this outbreak. It is 
not clear how lessons could be learned if 
they were not documented or 
communicated. 


15.74 In such situations, particularly 
given that the cause of the contamination 
was never found, the value of a 
lessons-learned review cannot be 
underestimated. It is in the interest of 
public health that all participants 
contribute to, and learn from, such 
reviews. 


Conclusion and 
Recommendations 


15.75 This nation-wide outbreak was 
one of the largest and most serious 
food-borne disease outbreaks in Canadian 
history. More than 800 cases across the 
country were investigated; over 80 percent 
of them involved children under 15 years 
old. Many suffered severe diarrhea, and at 
least 60 were hospitalized. Many more 
might have suffered had the various 
federal and provincial agencies not 
identified the contaminated product. 
However, many individuals could have 
avoided this illness had the federal and 
provincial health departments acted more 
swiftly and co-operatively. 


15.76 This case illustrates the need for 
formal arrangements that clearly set out 
the role and responsibilities of LCDC in 
relation to those of other participants in 
handling a nation-wide threat to public 
health. 


15.77 Other areas where improvement 
is needed include the ability of federal and 
provincial organizations to work together 
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co-operatively and to share information in 
a timely manner. Specifically, there is a 
need for prompt notification about a 
possible outbreak and a timely exchange 
of information to support an investigation 
into the outbreak. 


15.78 Health Canada should clearly 
identify, and communicate to other 
participants, who is to be called in the 
event of a suspected outbreak of 
food-borne disease and who is 
responsible for managing the response. 


Health Canada’s response: Implemented. 
The Roles and Responsibilities 
Framework, which establishes the specific 
accountabilities of Health Canada and the 
Canadian Food Inspection Agency, was 
revised in June 1999. 


In addition, Health Canada has developed 
a “Food-borne Illness Outbreak Response 
Protocol” in collaboration with the 
Canadian Food Inspection Agency and 
with provincial and territorial 
governments. The Protocol is a formal 
arrangement detailing Health Canada’s 
role and responsibilities in relation to its 
partners and will result in more 
co-ordinated and rapid responses to 
food-borne outbreaks. 


Operational procedures for Health 
Canada staff, with contact names and 
numbers and a flow diagram to aid in 
co-ordinating emergency responses, 1s 
being completed and will be available by 
mid-September 1999. 


15.79 Health Canada should work 
with provinces and territories to ensure 
the timely exchange of information and 
proper notification about 
interprovincial outbreaks of food-borne 
diseases. 


Health Canada’s response: Agreed; 
implementation is under way. The 
“Food-borne Illness Outbreak Response 
Protocol” provides for the rapid exchange 
and evaluation of information between 
parties to ensure prompt notification and 
identification of outbreaks. 
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In addition, in the context of developing 
the National Health Surveillance Network, 


federal/provincial/territorial deputy 


ministers of health have endorsed 
collaborative ventures such as the 
Canadian Integrated Public Health 
System (CIPHS), a project under the 
National Health Surveillance Infostructure 
(NHS1). A pilot phase of this initiative, 
designed to link laboratories with public 
health officials, will be implemented in 
British Columbia by 31 March 2000. 
When fully operational, this system will 
significantly reduce the time necessary for 
the recognition and communication of 
potential outbreaks. 


15.80 Health Canada and the 
Canadian Food Inspection Agency 
should carry out their respective roles 
and responsibilities collaboratively with 
other participants. 


Health Canada’s response: Agreed. As 
indicated in the response to the 
recommendation in paragraph 15.78, the 
implementation of the “Food-borne Illness 
Outbreak Response Protocol”, developed 
by Health Canada and its partners, will 
result in more co-ordinated and rapid 
responses to food-borne outbreaks. 


Canadian Food Inspection Agency’s 
response: Agreed. In partnership with 
Health Canada, the Agency updated the 
agreement on roles and responsibilities 
addressing food emergency response, in 
June 1999. In addition, the Agency, 
Health Canada and all provinces and 
territories have set out working 
procedures in the “Food-borne Illness 
Outbreak Response Protocol”, which 
enhances the response of the partnership 
to food emergencies. The Agency is also 
an active participant in the Canadian 
Food Inspection System Implementation 
Group, which seeks to harmonize federal, 
provincial and territorial food inspection 
systems. 


15.81 Health Canada and the 
Canadian Food Inspection Agency 
should ensure that they maintain 
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appropriate records of key meetings 
and decisions. 


Health Canada’s response: Implemented. 
Health Canada now routinely records 
minutes of all food-borne outbreak 
operational meetings as well as “lessons 
learned” from post-outbreak review 
meetings. 


Canadian Food Inspection Agency’s 
response: Agreed. The Agency has 
recently realigned responsibilities for food 
safety emergency response within the 
Office of Food Safety and Recall. One of 
the primary responsibilities of this Office 
is to maintain records of operational 
meetings and decision-making processes 
pertaining to food emergencies. 


15.82 Health Canada and the 
Canadian Food Inspection Agency 
should ensure that an appropriate 
review is conducted following an 
outbreak of food-borne disease. Such a 
review should include all participants 
involved in the handling of the 
outbreak, and lessons learned should be 
documented and communicated to all 
participants in a timely manner. 


Health Canada’s response: Agreed and 
implemented. The “Food-borne Illness 
Outbreak Response Protocol” provides 
that a post-outbreak review be conducted 
following national and international 
outbreaks or outbreaks that are unusual in 
nature involving a rare or unusual 
infectious agent. 


Canadian Food Inspection Agency’s 
response: Agreed. A framework is 
provided, within the Protocol, for the 
review of food-borne disease outbreaks in 
a collaborative manner. The review 
process described encourages the 
development of specific recommendations 
that will allow all participants and 
organizations to benefit through their 


application as appropriate. In instances 
where other partners are not involved in 
recalls, the Office of Food Safety and 
Recall undertakes internal reviews, with a 
view to improving Agency management of 
food emergencies, and thus reducing risk 
for Canadians. 


Health Canada’s overall comments: 
Health Canada’s review of this incident 
helped to identify areas for further 
improvement in the management of 
food-borne disease outbreaks, and the 
lessons learned are being applied. The 
experience of the episode has also 
contributed to the development of a 
national surveillance strategy, and the 
government has since announced, in the 
1999 Budget, a significant investment in 


food safety. 


Canadian Food Inspection Agency’s 
overall comments: The Canadian Food 
Inspection Agency is committed to 
enhancing the effectiveness of Canada’s 
food safety system and improving on the 
progress made in recent years to 
consolidate the various inspection systems 
into a single agency. This includes 
management of food-borne disease 
outbreaks. As noted in its responses to the 
preceding recommendations, the Agency 
has implemented changes to its 
management and operating procedures to 
address the concerns raised in this 
chapter. 


The Agency successfully managed 257 
food recalls in 1995-99, compared with 
165 the previous year. Many of these 
recalls involved a number of federal, 
provincial and municipal agencies. This 
multi-jurisdictional process plays a 
significant role in protecting the health of 
Canadians. The Canadian Food 
Inspection Agency is committed to 
working with its partners to contribute to 
safe food and to protect Canadian 
consumers. 
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y About the Audit 


Objectives 


The objective of the audit was to assess the adequacy of the management of a food-borne disease outbreak by 
Health Canada and the Canadian Food Inspection Agency, in collaboration with other jurisdictions. 


Scope 


The audit examined a 1998 nation-wide outbreak of food-borne disease and how it was managed by Health 
Canada and the Canadian Food Inspection Agency, in collaboration with provincial and local health 
authorities. Specifically, we examined the activities related to the identification and communication of the 
outbreak, the investigation to determine its source and cause, the recall and removal of the contaminated food 
product, and the post-outbreak review. We had extensive discussions with departmental staff at Health Canada 
and officials at the Canadian Food Inspection Agency as well as selected provincial health officials directly 
involved in the investigation and control of the outbreak. 


Criteria 
We expected that: 


e all participants would clearly understand the roles and responsibilities of Health Canada and the Canadian 
Food Inspection Agency in the event of a food-borne disease outbreak; 


e documented protocols and procedures would exist that indicate clearly what each participant should do 
when a national or interprovincial outbreak of food-borne disease occurs; 


e Health Canada’s surveillance systems for monitoring diseases would enable it to collect, analyze and 
disseminate all information necessary to help anticipate, prevent and respond to food-borne disease 
outbreaks; 


e all participants involved in managing and investigating an outbreak would work together, share 
information, and act quickly to protect the health of the public; and 


e all participants would contribute to, and learn from, post-outbreak reviews. 


Audit Team 


Assistant Auditor General: Maria Barrados 
Principal: Ronnie Campbell 
Director: Gerry Chu 


Julie Charron 
Anil Risbud 
Glenn Wheeler 


For information, please contact Ronnie Campbell. 
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Chapter 16 


Revenue Canada 


Goods and Services Tax: 
Returns Processing and Audit 


The audit work reported in this chapter was conducted in accordance with the legislative 


mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 


Accountants. The numbered paragraphs in bold face represent recommendations. 
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to registrants 
GST returns processing is being decentralized 


GST Pre-Payment Audit Program 
Pre-payment audits are an essential part of Revenue Canada’s GST enforcement activities 
Revenue Canada can do more to improve the effectiveness of its automated validity checks 
Delays in reviewing and auditing rejected refund claims add to interest costs and reduce service 
to registrants 
There is a need for up-to-date procedures to review refund claims 


GST Post-Payment Audit Program 
Better data and tools would improve Revenue Canada’s audit selection process 
More attention needs to be given to audit planning 
There are areas of strength in audit performance 
Some aspects of audit performance require improvement 
Support for the GST Audit Programs 
Staffing and training needs 
Information needs 
Control over audit files 


Reporting on Performance 


Conclusion 

About the Audit 

Exhibits 

16.1 Processing and Pre-Payment Audit — Summerside Tax Centre 
16.2 Pre-Payment and Post-Payment Audits — Tax Services Offices 
16.3 GST Administration at Revenue Canada 

16.4 Average Time to Process Documents — 1995-96 to 1998—99 
16.5 Fraud Case Illustrating the Importance of Pre-Payment Audits 
16.6 GST Credit Returns for 1997—98 

16.7 Example of a Good Post-Payment Audit 

16.8 Example of a Post-Payment Audit Requiring Improvement 


Page 


16—5 


16-7 
16-7 
16-7 
16-8 
16-9 
16-10 


16-10 


16-10 
16-10 
16-11 
16-12 


16-13 
16-15 


16-16 
16-16 
16-17 


16-19 
16-20 
16-21 
16-21 
16-23 
16-23 
16-24 
16-28 
16-28 
16-28 
16-30 


16-30 
16-30 


16-32 


16-8 

16-9 
16-10 
16-11 
16-17 
16-18 
16—24 
16—25 


Report of the Auditor General of Canada — September 1999 


16— 


‘@ 


» 


- ci -ou. ui a 
Siafs ime te a pardon gud) Gal riA-gy tel (mer © aT 7 
emt aha, wmubnettent RS 
hs s‘ntens'? oot Setents blaee ding vee airensh \_< 
curs Mba tal (Pv teed ieee etre ae |. © 
sinner tay Moa 16 ithientt to Lao ope Seite 
(2 Gare yal. Art eqqpety Tisy ee 3, Faas STs 
esberr vggsS tikes do, F940 att voy deeqesee 
toe eee ith Suillpee. 
Om miner) | 
pe) fies nove teanaa 
qonrs/hy lem griraqan 


intima 


items A sad) povied ds 
ott ia 


haliod? simmer pante ah cma ttn Laivexertt 1a 
HCO ior? Ot > hah ART el log brie 
ida ano] tai Gili eiaisatel, 7 ri © A 

O40) oe 2 | aeinamerd tower ob abil aries 
ale Toit 49 Do emnancnenl nal quill ots kell 
RO (OO! pp aevited dhe) Te) 

his st einn gi cr T ral wh) edgar 

cramer net Raruipe® Mande reat ed dat p Wralgeoal 


a — —— 


OUT toe umtopet — amern "Sp Sherman 


: 1 oe 


Revenue Canada 


Goods and Services Tax: 
Returns Processing and Audit 


Main Points 


16.1 We found that Revenue Canada’s Summerside Tax Centre, until recently its sole GST processing centre, 
has steadily become more efficient. It has streamlined several processes, enhanced its error correction procedures, 
reduced unit costs and improved the timeliness of its processing operations. As a prelude to decentralizing the 
processing of GST returns, Revenue Canada further simplified its processing procedures. The results of a pilot test 
point to further efficiency gains. 


16.2 At present, standards for returns processing tend to place insufficient emphasis on quality, accuracy, 
timeliness and unit cost, and the use of performance information is hampered by problems with the availability 
and reliability of data. Better performance measures would help to manage interest costs and improve service to 
registrants. 


16.3 Revenue Canada can do more to improve the performance of its set of automated validity checks. These 
checks are the cornerstone of its pre-payment audit program. However, they are not particularly discriminating in 
their initial targeting of inappropriate refund claims. Revenue Canada also needs to consider reviewing and 
auditing returns on which refunds are not requested at the same time as it conducts pre-payment audits of refund 
requests. 


16.4 Many of Revenue Canada’s post-payment audits consume too many staff hours and go on for too long. 
Revenue Canada needs to help its auditors shorten their audit times by providing closer supervision and by 
encouraging better planning, execution and control of audit work. Audit performance may also be improved by 
better selection of registrants for audit. 


Background and other observations 


16.5 The introduction of the Goods and Services Tax (GST) in 1991 had a huge impact on Revenue Canada’s 
activities. The Department had to develop a completely new processing operation and techniques to verify the 
accuracy of the returns submitted by almost 2 million registrants, ranging from small entrepreneurs to 
multinational corporations. 


16.6 In 1993, Revenue Canada opened a new tax centre in Summerside, Prince Edward Island to process all 
GST returns and rebate forms from non-Quebec registrants. (The ministére du Revenu du Québec administers 
GST processing and audit of Quebec registrants.) The Summerside Tax Centre employs the equivalent of about 
660 employees and has an annual operating budget of about $28 million. 


16.7 Revenue Canada now processes about 5.5 million GST returns each year, as well as 200,000 domestic 
rebate forms and 3 million remittances from GST registrants. 


16.8 The Department has begun to decentralize a portion of its GST processing activities to tax centres across 
the country. The processing of rebate forms will continue to be done in Summerside. 
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16.9 While, in general, GST returns processing is a tax centre activity, Revenue Canada’s audit programs are 
performed by auditors in tax services offices across the country. There are two major audit programs. The first is 
the pre-payment audit program, which accounts for about 25 percent of auditor time. After the data have been 
processed but before refunds are issued, Revenue Canada applies a variety of automated validity checks to ensure 
that requests for refunds are appropriate. The checks are designed to capture the requests with the highest risks of 
non-compliance. The captured refund claims are sent electronically to tax services offices across Canada for 
review and, in some cases, for audit. In 1997-98, the GST pre-payment audit program resulted in assessments of 
about $165 million. 


16.10 Revenue Canada also performs post-payment audits. These audits account for about 75 percent of auditor 
time and focus not only on particular returns but also on registrants with the highest risk of non-compliance. In 
1997-98, this work resulted in assessments of about $355 million. 


16.11 | We examined the efficiency and effectiveness of the GST returns processing and audit functions. The 
efficient and effective use of resources is important because it has a direct impact on protecting the public purse 
and improving service to registrants. 


Revenue Canada’s responses to our recommendations are included in this chapter. The Department is in 
agreement with the seven recommendations, and its responses describe a number of actions that have been 
completed or are in progress to deal with them. 
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Introduction 


GST in general 


16.12. The GST is a multi-stage sales 
tax levied on taxable goods and services. 
“Multi-stage” means that the GST applies 
on each exchange of taxable goods and 
services, from the initial producers/ 
suppliers through intermediate 
producers/suppliers to consumers. To 
ensure that the value added at each stage 
of the supply chain is taxed only once, a 
mechanism exists by which suppliers can 
obtain a refund of GST paid on their 
purchases. (Suppliers of exempt goods and 
services are exceptions to the rule; they 
are effectively treated as consumers.) A 
claim for recovery of GST paid on a 
business input 1s known as an “input tax 
credit.” 


16.13. GST registrants must submit 
returns to Revenue Canada on a monthly, 
quarterly or annual basis, depending on 
their sales volumes. GST charged to 
customers and input tax credits claimed on 
purchases are both recorded on the return. 
If the total GST charged by a registrant 
exceeds the input tax credits claimed, the 
registrant is expected to remit the excess 
to Revenue Canada. This is known as a 
“debit return.” If input tax credits claimed 
exceed the GST charged, the registrant 
seeks a refund from Revenue Canada. 
This is known as a “credit return.” 


16.14 Depending on the nature of their 
operations and their circumstances, 
registrants may also be eligible for various 
rebates. For example, municipalities are 
eligible for a rebate designed to lower the 
effective rate of GST applied to them. 
Rebate entitlements may be included in 
GST returns and, if they are large enough 
to offset the net GST otherwise payable, 
they will produce a credit return. In other 
cases, rebate entitlements are claimed on a 
separate form. 


16.15 In May 1999, there were 
approximately 2,411,000 active GST 
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registrants. Revenue Canada administers 
the GST for the approximately 
1,925,000 registrants outside Quebec. 
Under a 1991 agreement between the 
Government of Canada and the 
Government of Quebec, the ministére du 
Revenu du Québec administers the GST 
on behalf of Revenue Canada for 


Revenue Canada 


approximately 486,000 registrants in administers the GST 
Quebec. The ministere uses Revenue for the approximately 
Canada’s guidelines and policies but 4 
produces its own audit strategies and 1,925,000 registrants 


administrative procedures. The GST 
administered by the ministére is recorded 
in Revenue Canada’s databases. 


outside Quebec. 


16.16 The Harmonized Sales Tax 
(HST) collected in three of the Atlantic 
provinces is an integral part of the GST 
program. For simplicity, in this chapter the 
term GST also includes HST. 


Returns processing 


16.17 The processing of GST returns 
and rebate forms begins when incoming 
mail is opened and sorted by type of 
document (see Exhibit 16.1). At this point, 
the documents are identified and checked 
for completeness and special conditions, 
such as attached correspondence and 
payments. The returns and forms are 
placed in batches and sent for document 
preparation and data capture. Information 
from the returns and forms goes into the 
GST mainframe computer system. The 
documents with errors go to the error 
corrections section. There, to the extent 
possible, errors such as incomplete 
returns, incorrect registrant names and 
business numbers, incorrect reporting 
periods, and mathematical errors are 
corrected. At times, in order to obtain 
correct information, departmental officers 
contact the registrant. 


Revenue Canada has 
begun to decentralize 
GST processing. 


16.18 Until recently, Revenue Canada’s 
Summerside Tax Centre was responsible 
for processing all GST returns and rebate 
forms sent in by non-Quebec registrants. 
Revenue Canada has begun to 
decentralize a portion of its GST 
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processing activities to tax centres across 
the country. Rebate forms will continue to 
be processed in Summerside. 


16.20 Automated validity checks select 
about 30 percent of the credit returns and 
rebate forms. These items are 
automatically referred to tax services 


Automated validity Pecceacuaneenite offices across Canada according to the 

l b Boy, registrant’s location (see Exhibit 16.2). 
checks select about There, a departmental officer brings up on 
30 percent of the 16.19 After data from GST credit a computer screen information about the 


credit returns and 
rebate forms; about 


returns and rebate forms have been 
captured in the mainframe computer 
system, a series of automated validity 
checks is applied to the data (see 


selected items. The officer examines each 
item, and considers such factors as 
industry type, prior filing history and prior 
audit results. The officer then decides 


97 percent of the Exhibit 16.1). These checks are intended whether to pay the amount claimed (called 
items selected are to detect credit returns and rebate forms “waiving” the item) or to conduct a more 
: that may not comply with legislation or in-depth examination. About 97 percent of 

subsequently waived. that may warrant investigation for some the items are waived. Generally, for the 

other reason. In other words, automated approximately 3 percent of items 

validity checks are based on the requiring further examination, an officer 

Department’s assessment of risk contacts the registrant to ask for more 

associated with credit returns and rebates, information and/or documentation. If 

and reflect its policy of periodic scrutiny. satisfied, the officer approves the item for 
Exhibit 16.1 


Processing and Pre-Payment Audit - Summerside Tax Centre 


FROM 
REGISTRANTS 


PROCESSING 


PRE-PAYMENT 


Remittances 


Credit Returns 


[ 
Bank Deposits | 


Debit Returns 


Rebate Forms 


Review and correction of such matters as incomplete returns, incorrect 
registrant data, incorrect reporting period, and mathematical errors 


registrant database 


Credit return, rebate form and debit return information entered into the 


Automated validity checks applied to 


AUDIT credit returns and rebate forms only 

Returns/rebates not selected by Returns/rebates selected by the 

the automated validity checks automated validity checks 
Public Works and Government Credit returns and rebate forms 
Services Canada instructed to sent electronically to the 
produce cheques and send to applicable tax services offices 
registrants for further review 
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payment; if not satisfied, the officer 
adjusts or rejects the item. 


Post-payment audits 


16.21  Pre-payment audit activity is 
aimed at preventing inappropriate 
payment of refunds claimed on credit 
returns and rebate forms. This means that 
the activity is focussed on detecting 
overstated or ineligible input tax credits 
and rebates. However, Revenue Canada 
also wishes to find other kinds of 
non-compliance with the many 
requirements of the Excise Tax Act, and to 


PRE-PAYMENT AUDITS 
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find problems with refunds that were paid 
in the past. This is the purpose underlying 
the post-payment audit program. 


16.22 Post-payment audits are handled 
by the tax services offices (see 

Exhibit 16.2). Relying heavily on their 
knowledge of the registrant population, 
external sources, leads from other areas of 
the Department — for example, 
pre-payment audit or income tax — and 
analysis of departmental databases, teams 
of post-payment auditors select registrants 
for examination. They normally select 
registrants who are expected to represent a 
high risk of non-compliance. 


Exhibit 16.2 


Pre-Payment and Post-Payment Audits — Tax Services Offices 


POST-PAYMENT AUDITS 


{ . co 
| Credit returns and rebate forms sent 


Registrant selected for post-payment audit 
as either a national or local initiative 


SOURCE/ electronically to the applicable tax 
INPUT | services offices for further review 
AUDIT Tax Services Offices — Decisions 


WORKLOAD | * 
audit); or 
e Pre-payment audit required 


Waive (approve for payment without 


Tax Services Offices — Decisions 


Determine scope of work and develop 
audit program 


a 


Approved for | 
payment 


Public Works and Government 
Services Canada instructed to 
produce cheques and send to 


registrants 
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Auditors review specific 
| documentation, perform 
selected audit procedures, etc. 


Approved with 
no change 


y & 


Audit 


Rejected/ 
Adjusted 


A, REN 


Notice of assessment sent to 
(where applicable) 
: | 


registrant 


= 


16-9 


Revenue Canada — Goods and Services Tax: 
Returns Processing and Audit 


16.23. In 1997-98, the GST 
pre-payment and post-payment audit 
programs resulted in assessments of about 
$520 million. The net revenue assessed 
was approximately $624 per direct audit 
In 1997-98, the GST pis 
audit programs 


resulted in Focus of the audit 


assessments of 16.24 Exhibit 16.3 illustrates a 

about $520 million. simplified GST administration framework 
patterned on Revenue Canada’s official 
activity structure. Our audit examined the 
efficiency and effectiveness of Revenue 
Canada’s GST returns processing and 
audit activities. Further details are found 
at the end of the chapter in the section 
About the Audit. 


Observations and 
Recommendations 


GST Returns Processing 


GST processing centre has improved its 
efficiency 


16.25 The GST document processing 
facility, located in the Summerside Tax 
Centre, started operations 1n October 
1993. In 1998-99, the Centre processed 
about 5.5 million GST returns, 

200,000 domestic rebate forms, and 

3 million remittances. It employs the 
equivalent of about 660 full-time 
employees, and costs about $28 million a 
year to operate. This includes the 


Exhibit 16.3 Facilitation 


GST Administration at 


rulings and answers to inquiries 
Revenue Canada 


Provides forms, interpretation bulletins, advance 


Collections 


Collection of GST owing 


database of registrant 
information 


e remittances 

e GST returns 

e rebate forms 
refunds requested 


with their obligation to 
report complete and 
accurate information and 
calculate their tax lability 


Registrant Database Processing Audit Policy and Legislation 
Maintenance of complete, Prompt and accurate Audit to ensure that : Issuing of policies and 
up-to-date and accurate processing of: registrants are complying | assistance in developing 


and amending legislation 


tL 


Dispute Resolution 


Review of notices of 
objection and handling of 
appeals to courts 


Monitoring and Reporting 


Maintenance of appropriate information systems for use by departmental 
officials and for reporting to Parliament on GST administration 


Our audit dealt primarily with the shaded boxes. We did not audit the processing of remittances. 
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equivalent of about 65 full-time 
employees and $1.4 million for income 
tax and excise tax processing. 


16.26 GST registrants have to file 
returns monthly, quarterly or annually, 
depending on their sales volumes. Returns 
are generally due toward the end of a 
month, resulting in 12 peak volumes a 
year, with higher peaks for the quarterly 
and annual periods. To accommodate this 
fluctuation, the Department must be 
flexible when allocating resources. We 
found that the rate at which documents are 
processed varies during the year but, as 
noted in Exhibit 16.4, about 90 percent of 
the documents get processed within 

16 days and most of the remaining 
workload is processed in the next 45 days. 
The percentage of transactions processed 
within 16 days has generally increased. 


16.27. Managers and employees in 
Summerside and at headquarters have 
worked together to streamline several 
workflows. Efficiency gains have been 
made in several areas: mail extraction 
processes have improved; redundant 
document preparation work has been 
eliminated; data capture methods have 
been improved; and error correction 
procedures have been enhanced. 
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16.28 We calculated that the unit cost 
of processing a standard return that has no 
errors fell from about $0.66 in 1995-96 to 
about $0.37 in 1998-99. Similarly, for 
returns accompanied by a payment but 
containing errors, the unit cost of 
processing fell from approximately $3.48 
in 1995—96 to about $1.91 in 1998-99. 
Taking 1998—99 document volumes and 
using the 1995—96 and 1998-99 unit costs, 
we calculated that efficiency gains 
between the two years amounted to about 
$2 million. 


Efficiency gains have 
been made in several 
areas. 


The use of performance measures and 
benchmarking for GST processing is 
limited 


16.29 Good management of GST 
returns processing involves handling a 
given volume of documents according to 
specified standards for quality, accuracy, 
timeliness and unit cost. At present, 
Revenue Canada has a lot of historical 
information on volumes of production, but 
less data on quality, accuracy, timeliness 
and unit cost of processing. Although 
information on cost and production 
quantity exists in tax centres and at 
headquarters, management does not take 
full advantage of it to optimize the use of 
resources and further improve efficiency. 


Percentage Completed Exhibit 16.4 
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Management does not 
take full advantage of 
available information. 


Since 1993, Revenue 
Canada has not 
undertaken 
benchmarking 
activities with 

other large 
processing centres. 


16.30 About 40 performance standards 
were recently established for processing 
operations at the Summerside Tax Centre. 
However, only volume output information 
is used. Very few efficiency measures 
have been developed and reported. Also, 
standards deal mostly with units 
completed per hour but do not specify the 
expected level of quality, such 

as percentage of errors and percentage 
completed in a specified time period. 
Another shortcoming of some standards is 
that they may be aggregated to a point 
where relevant information is lost. For 
example, the focus is on the number of 
documents in the work-in-progress file as 
opposed to, say, the age of the 
work-in-progress items at peak and 
non-peak times for given types of 
transactions. 


16.31 Since 1993, Revenue Canada has 
not undertaken benchmarking activities 
with other large processing centres to 
assess the quality of its practices and 
technologies. The performance standards 
have not been compared with similar 
measures used in other government 
processing operations such as the 
ministere du Revenu du Québec, or in 
private industry. In the context of the new 
Canada Customs and Revenue Agency, 
benchmarking against similar 
organizations could indicate whether some 
service delivery options — such as 
contracting out — would further improve 
efficiency. 


16.32 Revenue Canada should update 
its GST returns processing standards to 
include quality, accuracy, timeliness and 
unit cost, and should measure 
performance accordingly. Where 
appropriate, standards should 
differentiate between peak and 
non-peak times and among various 
types of transactions. Standards should 
presume the use of efficient processing 
procedures and, where possible, 
performance should be benchmarked 
against similar organizations. 
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Department’s response: The Department 
fully agrees with the importance of 
standards for returns processing. In 
1997-98 the Department began a project 
to significantly improve GST/HST returns 
processing and establish performance 
measures using the Balanced Scorecard 
approach. Existing processes were 
compared with best practices in other 
revenue lines within Revenue Canada and 
concepts were imported from elsewhere, in 
particular the manufacturing industry. 
These improved processes are captured in 
the revised GST/HST Accounting and 
Processing Procedures (GAPP) manual 
that describes how processing is now 
being performed. As noted in 

paragraph 16.54, these changes have 
already produced measurable 
improvements. 


Key activities were identified as part of 
the development of the new GAPP, and as 
noted in paragraph 16.30, about 

40 performance standards have been 
established. The next step will be to 
develop a Balanced Scorecard of 
standards, including quality, accuracy and 
timeliness. The first task will be to confirm 
the desired quality and accuracy 
characteristics through a survey of client 
needs, and benchmarking of performance 
levels attained by other revenue 
administrations and private industry. It 
should be noted that the move of GST/HST 
processing to multiple tax centres provides 
the benefit of closer internal 
benchmarking against other revenue lines 
and between centres. 


The Department will carefully take this 
recommendation into consideration as it 
continues its development of a balanced 
set of performance standards for GST 
processing. Service standards and actual 
performance will be published regularly. 


Reliability and timeliness of 
performance information are deficient 


16.33. Having reliable and up-to-date 
data is crucial to managing a high-volume 
processing function like the Summerside 
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Tax Centre. Managers receive data by 
activity on labour-hour usage, production, 
and cost compared with budget. 
Information comes from various systems 
that operate on different cut-off periods. 
Information on labour hours and 
production quantity is captured 
simultaneously in ARMS (A Resource 
Management System). Salary and other 
operating expenses are initially captured 
in other systems and are then reflected in 
ARMS after a three- to five-week delay. 
ARMS information is most reliable at 
year-end when all data have come in. 


16.34 We found that there can be wide 
discrepancies in production information 
for a given year across systems, and over a 
number of years. For example, the ARMS 
production statistics for the months of 
August 1997, April 1998 and May 1999 
showed no production even though the 
Summerside Tax Centre was operating 
during those months. Such discrepancies 
are due to inconsistencies in data input 
between systems at Summerside, and are 
not corrected because the data contained 
in ARMS are not reconciled with data 
contained in Summerside’s local system. 
Management is aware of the discrepancies 
between the systems. 


16.35 To establish a unit cost, we relied 
primarily on data supplied by 
management at the Summeside Tax 
Centre, since these data come directly 
from automated systems that form part of 
Summerside’s regular reporting systems. 
The data do not have the large time lags 
that the ARMS data have. 


16.36 Revenue Canada is completing 
the phased implementation of a Common 
Administrative System, which will 
eventually replace the Department’s entire 
Resource Management Information 
System, of which ARMS is a component. 
It is still uncertain how many of the 
currently available financial reports the 
new system will be able to deliver 
immediately. The Department plans to 
temporarily maintain certain processing 
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and reporting components of ARMS to 
ensure that the majority of currently 
available and useful financial reports will 
continue to be accessible. 


Better performance measures will help 
manage interest costs and improve 
service to registrants 


16.37. The prompt processing of GST 
returns is a service to registrants. It also 
reduces the government’s interest costs. 
The Excise Tax Act requires that interest 
be paid to registrants on amounts owed 
beginning 21 days after the receipt of a 
credit return. Similar rules apply to rebate 
claims. According to departmental 
records, in 1997-98 the interest charges 
incurred by Revenue Canada for late 
payment of refund and rebate claims 
amounted to about $13 million, about 
$3.2 million of which related to 
transactions processed by the ministére du 
Revenu du Québec. 


There can be 

wide discrepancies 
in production 
information. 


16.38 Monitoring the performance of 
returns processing operations requires a 
variety of performance measures. One of 
these measures ought to be the amount of 
interest the Department has paid out each 
year on overdue refunds, compared with a 
target amount. However, the Department 
has not set a target for what it would 
consider a reasonable interest cost of 
administering GST. 


16.39 Not all of the interest cost is 
attributable to the Summerside Tax 
Centre. As well as the number of days it 
takes to process the documents at the 
Centre, interest costs are influenced by the 
number of days required for the tax 
services offices to review refund claims 
rejected by the automated validity checks 
(see paragraphs 16.70 to 16.73) and the 
number of days required for Public Works 
and Government Services Canada to issue 
the cheques. Nevertheless, any reduction 
in the time the Summerside Tax Centre 
takes to process its documents would 
contribute to reduced interest costs and 
improved service to registrants. 
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Improving the 
processing of 
documents can reduce 
costs and improve 
service to registrants. 


16.40 The Department has carried out 
studies on how to achieve its turnaround 
standard for processing documents while 
maintaining workload at a consistent 
level, but it has not analyzed its 
processing activities specifically to find 
ways to reduce interest costs. 


16.41 Aside from generally improving 
the timeliness of processing, one way to 
reduce interest costs would be to give 
priority in processing and error correction 
to the large refund claims; these result in 
high interest costs if they are not 
processed speedily. This would be 
consistent with the current practice of 
giving high priority in the mail extraction 
procedures to cheques over $25,000. 


16.42 Using data supplied by Revenue 
Canada, we found that in 1997-98, 
refunds were issued to registrants on about 
2 million credit returns and rebate forms. 
The Department estimates that about 

30 percent of these refunds included 
interest. The actual interest expense for 
the year was $9.8 million, or about $16.33 
per cheque on which interest was paid. In 
the management of items awaiting data 
capture and in the processing of mail 
received from tax services offices, we 
calculate that, as a rough order of 
magnitude, there is scope to reduce 
interest costs by between $800,000 and 
$4 million per year. 


16.43 Improving the processing of 
documents can reduce not only interest 
but also other costs, and can improve 
service to registrants. For example, after a 
set date following the end of the filing 
period, the GST system automatically 
generates notices to registrants who have 
not filed their returns. Even if returns have 
been received, if they are not processed 
within this timeframe the system 
automatically generates the notices and 
incurs additional expenses for printing and 
mailing. This also results in increased 
salary costs for client service officers at 
the Summerside Tax Centre and at tax 
services offices (who must respond to 


phone inquiries from registrants) and error 
correction staff at Summerside (who must 
deal with second copies of returns 
submitted by registrants). 


16.44 While reducing interest costs is a 
valid objective, it must be balanced 
against the potential monetary loss 
associated with the payment of improper 
refund claims if adequate time is not 
allowed for examination of the claims. 


16.45 Revenue Canada should 
monitor and control the amount of its 
portion of interest charges, and should 
work with Public Works and 
Government Services Canada to find 
ways to better manage interest costs and 
improve service to registrants. 


Department’s response: The Department 
agrees with the need to monitor and 
control interest costs. Interest costs are 
incurred when all stages of the process — 
returns processing, pre-payment audit and 
cheque production — take longer than the 
time provided for in the Excise Tax Act. 
Because interest costs are the result of 
processing delays and not their cause, the 
Department believes that processing time 
should be the primary performance 
indicator and thus the focus of attention 
for process improvements. Concentrating 
directly on the reduction of interest costs 
could unfairly skew the focus of 
processing; for example, the rapid 
handling of large-business returns would 
become more important than 
small-business returns. The Department 
will therefore control the level of interest 
costs by focussing on improved timeliness 
and quality for all business clients so that 
a fair allocation of the benefits is 
achieved. 


The Department continues to seek process 
improvements that will reduce the initial 
processing time, and the time taken to 
produce the cheque. Improvements in 
these two areas will allow more adequate 
time for consideration of the validity of 
the claim. Taken together, these 
improvements will reduce the risk of 
improper payments and the total time 
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taken to process returns. The results of 
returns processing improvements to date 
are already evident, as reported in 16.54, 
and work will continue in this area. The 
Department also continues to work with 
Public Works and Government Services 
Canada (PWGSC) to reduce the time 
taken to process payments, and significant 
progress has already been recorded. For 
example, Revenue Canada now provides 
PWGSC with daily payment files, and 
cheques and direct deposits are produced 
within three business days. In the near 
future, PWGSC will offer electronic data 
interchange payments that will provide 
next-day deposit into the client's bank 
account. 


GST returns processing is being 
decentralized 


16.46 In October 1996, the Department 
announced its decision to “realign” its 


processing of income tax and GST returns. 


Under this initiative, the Ottawa Tax 
Centre will become a national call centre 
and payment-scanning facility. The 
personal and corporate income tax returns 
processed by the Centre are being 
distributed among the remaining tax 
centres. Also under the initiative, GST 
processing 1s being decentralized: some 
document handling functions now 
performed solely by the Summerside Tax 
Centre will be done at other designated 
tax centres. Summerside will keep some 
GST processing and will begin processing 
income tax returns. 


16.47 We did not audit the entire 
realignment process. However, we 
understand that Revenue Canada decided 
to reduce the number of tax centres that 
were processing income tax returns 
because the introduction of electronic 
filing of personal income tax returns and 
other initiatives had left an excess of 
processing capacity. 


16.48 Revenue Canada officials 
informed us that the rationale for partially 
decentralizing GST processing when it 
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realigned income tax processing was 
management’s aim to provide in future a 
single tax form combining all types of 
instalments and a GST return, and a single 
statement of account for all taxes. 
Decentralization would also allow each 
taxpayer to deal with only one tax centre. 
The Department also wished to ensure that 
each tax centre, except for Ottawa, would 
maintain employment at approximately 
the current level. 


16.49 The Department has told us that 
it did not perform a cost-benefit analysis 
of the available alternatives for 
decentralizing GST returns processing. It 
wished to put in place a fairly uniform 
organizational structure among tax 
centres. This would facilitate the 
management of all tax streams. 


16.50 We expect that there will be some 
negative consequences of decentralizing 
GST returns processing. These 
consequences include initial start-up costs 
such as purchasing additional equipment 


The Department did 
not perform a 
cost-benefit analysis 


and training staff. Also, the processing of of the available 
GST rebate forms is to remain at : 
Summerside. This means that registrants alternatives for 
who file regular GST returns and rebate decentralizing GST 


forms will have each document processed 
at a different location. This could cause 
difficulties in matching documents, 
increased costs to the registrants and, if 
there are delays, increased interest costs 
on overdue refunds and rebates. 


returns processing. 


16.51 Decentralizing GST processing 
means that functions will be performed at 
several sites across the country, with lower 
volumes than at Summerside. Moreover, 
should electronic filing of GST be 
introduced, the volume of work would 
decrease further. This raises the possibility 
that unit costs of processing may increase. 
Departmental officials informed us that, to 
deal with the possibility, they conducted a 
thorough review of Summerside’s 
procedures for processing GST returns and 
developed a plan to simplify the activity 

in a way that could be transplanted to 
other tax centres. This meant, for 
example, finding substitutes for certain 
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The pre-payment audit 
program can detect, 
on a timely basis, 
non-compliance by 
registrants. 


local area network-based computer 
applications that at present can be run 
only at Summerside. 


16.52 A key aspect of simplifying the 
process was to pave the way for “lean 
production” techniques that can achieve 
low unit cost in an environment 
characterized by small volumes of a 
variety of transactions. This will require 
each tax centre to develop innovative 
processing techniques to deal with unique 
situations it may face and, where 
appropriate, to share such “best practices” 
with other tax centres. 


16.53. In November 1998, the 
Department established a GST pilot 
project at Surrey Tax Centre. This project 
was not intended to determine whether or 
not decentralization ought to be 
undertaken. The project was intended 
solely to iron out the practical difficulties 
of transplanting processing operations. 


16.54 To determine the likely impact of 
decentralization on processing costs, we 
compared the unit cost of two activities — 
data capture and error correction — in the 
Summerside Tax Centre for the period 
April 1998 to October 1998 (before the 
pilot project commenced) with the unit 
costs at both Summerside and Surrey 
during the pilot for the period April 1999 
to June 1999. Our comparison indicates 
that hourly production rates were higher in 
both tax centres during the pilot than 
Summerside had achieved previously. 
This improved performance may be due to 
Revenue Canada’s efforts to simplify 
processing. 


GST Pre-Payment Audit Program 


16.55 In general, GST returns 
processing is a tax centre activity. It 
consists of performing relatively routine 
procedures on documents submitted by 
registrants. Registrants are rarely 
contacted, and data are not usually 
verified in detail. By contrast, Revenue 
Canada’s GST audit programs are 
performed by auditors in tax services 
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offices across the country. The work 
involves verifying the data submitted by 
registrants. This typically means obtaining 
additional information, often by visiting 
the registrant. 


16.56 The pre-payment audit program 
has been in existence since the GST 
began. It is intended to prevent the 
inappropriate payment of refunds claimed 
on credit returns or rebate forms. Refund 
claims are subjected to a series of 
automated validity checks. A claim that 
fails one or more checks is referred 
automatically to the appropriate tax 
services office. There, a departmental 
official reviews the claim and decides 
either to waive it (in which case the claim 
will be paid) or to have it subjected to an 
in-depth review and possibly a 
pre-payment audit. 


Pre-payment audits are an essential 
part of Revenue Canada’s GST 
enforcement activities 


16.57. The pre-payment audit program 
can detect, on a timely basis, 
non-compliance by registrants resulting 
from, for example, bookkeeping errors, 
duplicate refund requests, a 
misunderstanding of one of the numerous 
requirements of the Excise Tax Act, or 
fraud. While post-payment audits can 
detect the same types of non-compliance, 
registrants may receive refund payments 
for several years before being selected for 
such an audit and only a small percentage 
of registrants undergo post-payment 
audits. Furthermore, a post-payment audit 
may occur too late to recover 
inappropriate refunds. This would be the 
case if, for example, the registrant had 
made fraudulent refund claims and had 
ceased to be in business before a 
post-payment audit was conducted (see 
Exhibit 16.5), or if the registrant were to 
experience financial difficulties during the 
post-payment audit. 


16.58 During our visits to tax services 
offices, officials told us that pre-payment 
audits are a significant source of leads for 
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Revenue Canada can do more to 
improve the effectiveness of its 
automated validity checks 


future GST and income tax audits. Our 
sample of post-payment audits (see 
paragraph 16.77) indicated that the largest 
identifiable source of these audits was the 
pre-payment audit program. 16.61 As indicated in Exhibits 16.1 

and 16.2, two steps are taken to select 
refund claims with the highest risk of 
error. First, automated validity checks are 
applied to the credit returns and rebate 
forms being processed by the Summerside 
Tax Centre. Second, tax services offices 
review the items rejected by the checks to 
determine which should be waived and 
which should be audited. 


16.59 As an indication of how effective 
pre-payment audits are at detecting 
non-compliance, consider these results. In 
1997-98, pre-payment audits prevented 
inappropriate payments at the rate of 
$1,518 per direct audit hour. These 
included large refund claims that had been 
requested by some well-known and 
established businesses, and that were 16.62 
found to be inappropriate and payment 
prevented after only a few hours of work. 
The $1,518 per hour represents a net cash 
saving: since the pre-payment audits 
prevent payments from being made, there 
is no potential for bad debts. 


The automated 
validity checks are 
not particularly 
discriminating in their 
initial targeting of 
inappropriate refund 
claims. 


A high percentage of credit 
returns and rebate forms continue to be 
rejected by the automated validity 
checks. Approximately 30 percent of the 
credit returns are rejected by the 
automated validity checks (see 

Exhibit 16.6). This indicates that the 
checks are not particularly discriminating 
in their initial targeting of inappropriate 
refund claims. If the checks were refined 


16.60 Weare pleased to note that 
Revenue Canada is increasing its 


pre-payment audit activities. Resources 
allocated to the program in 1999-2000 


have increased by 9 percent over 1997-98. 


In addition, the Department has initiated 
an enhanced compliance initiative, the 
main focus of which is to identify and 


to better target the high-risk credit returns 
and rebate forms, and if the tax services 
offices could better determine which 
rejected returns were most likely to be in 
error, the offices could likely reduce the 
time spent reviewing the rejected returns. 


address potentially abusive refund claims. The saved time could be used to audit 


The following case is based on a real case of GST fraud currently being pursued by Revenue Exhibit 16.5 


Canada. It illustrates the type of GST fraud that Revenue Canada uncovers, and the importance of 


a well-performing pre-payment audit program. Fraud Case Illustrating the 


Importance of Pre-Payment 
Several individuals set up corporations and registered them for GST. The corporations then filed Audits 
credit returns — the input tax credits claimed exceeded the reported GST revenue. Some of the 
credit returns were rejected by Revenue Canada’s automated validity checks, reviewed by auditors 
and approved for payment. In total, over $20 million in GST refunds was paid to the corporations. 


The filing of credit returns was not unusual given the stated nature of the corporations’ business. 
However, in reality there were no purchases or sales, and the refund claims were fraudulent. 


The fraud came to light when one of the corporations was selected for a post-payment audit. 
Because of the way the individuals structured the corporations’ affairs, Revenue Canada has not 
yet identified any assets that can be used to recover the amounts paid out. Its investigations are 
continuing. 


We believe that one of the best ways for Revenue Canada to deal with schemes such as this is to 
prevent the refund cheques from being issued in the first place. This is the primary purpose of the 
pre-payment audit program. 
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additional returns or to perform more rejected by the checks. Other ways to 

detailed audit work on the ones rejected. make the current checks more effective 
include: 

16.63 In 1994 we noted that little 

analysis had been done on the usefulness e analyzing the amount of revenue 

of each automated validity check. Since assessed from each check to determine 

then, Revenue Canada has made some which checks are the most effective in 

refinements to the checks. Some were detecting amounts that should not be 

made in response to legislative changes, refunded; and 


such as the introduction of the HST. 

Others were made to correct errors in the 
There may be more computer logic or to improve the cost 

at . effectiveness of specific checks. However, 

efficient and effective there has still been little analysis of the 
checks that Revenue relative cost effectiveness of each check 
for the purpose of reducing the number of 
refund claims being rejected by the 
system. The percentage of rejected returns 16.65 Other automated validity 
is approximately the same as in 1992-93. checks may be more efficient and 
effective. While the current checks reject 
many credit returns and rebate forms, it is 
possible that they are not detecting some 
high-risk refund claims. There may be 
more efficient and effective checks that 
Revenue Canada could use. To help 
identify these, Revenue Canada could 


e introducing a point-scoring system to 
give a weighting to each check. This 
system could be used both to reduce the 
number of rejected items — they would 
need to exceed a certain point score to be 
rejected — and to better indicate which 


Canada could use. rejected refund claims should be audited. 


16.64 The automated validity checks 
used most often are those that determine 
whether the amount claimed by a 
registrant on a return (or the cumulative 
amount claimed on a series of returns) 
exceeds a pre-determined threshold 
amount. Increasing these thresholds for 


: ; 2 consider: 
the registrants with the lowest risk of 
non-compliance might significantly e analyzing the results of post-payment 
reduce the number of refund claims being audits to identify refund claims that 
Exhibit 16.6 Amount 
Number Percentage ($ millions) Percentage 
Gey ced nets tor 123) 0 eee (A)| 1,851,000 100.0 18,379 100.0 
Less: automatically approved 
at the Summerside Tax Centre (B)} 1,325,000 71.6 7,619 41.5 
Rejected by automated 
validity checks (©) 526,000 28.4 10,760 58.5 
Less: Waived for payment (D) 509,000 DAS) 9,854 53.6 
Audited (B) 17,000 9 906 4.9 
Dollars assessed from audit (F) ILS37 
Percentage waived 
of those rejected [(D) = (C)] 96.9 91.6 
Percentage audited 
of those rejected ((E) = (C)] 3.1 8.4 
Percentage assessed amount 
Source: Revenue Canada of audited amount [(F) + (B)] | MS 
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passed the checks but were subsequently 
found to have errors; 


e developing checks based on 
registrant profiles; and 


e selecting credit returns and rebate 
forms at random for audit, thus helping to 
determine levels of compliance and 
possibly identifying new checks that 
should be added to the system. 


16.66 Debit returns are not part of 
the pre-payment audit program. Returns 
where the GST charged to customers by 
the registrant exceeds the input tax credits 
claimed by the registrant are referred to as 
“debit returns.” While these returns do not 
result in refunds to registrants, they may 
contain as much (or more) understated 
GST revenue or overstated input tax 
credits as credit returns. Currently, such 
errors are not identified until a 
post-payment audit is conducted. 
However, only a small percentage of 
registrants undergo post-payment audits, 
and these may be performed several years 
after an erroneous return has been filed. 


16.67 One way that Revenue Canada 
could identify incorrect debit returns at an 
earlier date would be to develop 
automated validity checks for these 
returns. Her Majesty’s Customs and 
Excise in the United Kingdom is currently 
using a limited number of such checks to 
select debit returns for review and audit, 
and is planning to extend the use of these 
checks in the future. 


16.68 Another way for Revenue Canada 
to identify incorrect debit returns at an 
earlier date would be to expand its 
Computer Assisted Audit Selection 
(CAAS) system (see paragraphs 16.82 

to 16.86) to select debit returns for review 
and audit at the same time as it conducts 
pre-payment audits of credit returns. 
Revenue Canada officials have informed 
us that the ministere du Revenu du 
Québec already has such a program in 
place. 
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16.69 Revenue Canada should 
enhance its ongoing program to review 
and revise the automated validity 
checks. It should also consider 
reviewing and auditing debit returns at 
the same time as it conducts 
pre-payment audits of credit returns. 


Department’s response: The Department 
agrees with the recommendation to 
enhance its ongoing program to review 
and revise the automated validity checks. 
As part of this process, the Department 
will carefully examine the Auditor 
General's suggestions in paragraphs 16.64 
and 16.65. Additionally, the Department 
intends to modify the automated validity 
checks to include the appropriate risk 
issues identified in the Computer Assisted 
Audit Selection (CAAS) system. 
Furthermore, recognizing that one of the 
key ways in which to prevent fraudulent 
refund claims is to ensure that at the time 
of registration only legitimate businesses 
are registered for GST, departmental staff 
are working on a number of initiatives in 
this area. 


Currently, errors in 
debit returns are not 
identified until a 
post-payment audit 
is conducted. 


With respect to debit returns, it should be 
noted that the Department already 
assesses all debit returns for risk using 
CAAS and the Department advocates the 
continuation of its work to further improve 
the CAAS system for debit returns. This 
notwithstanding, the Department will 
examine ways to conduct earlier reviews 
and audits of registrants identified as high 
risk. Senior departmental officials will be 
meeting with officials from other 
Jurisdictions to share audit strategies and 
approaches for both credit and debit 
returns. 


Delays in reviewing and auditing 
rejected refund claims add to interest 
costs and reduce service to registrants 


16.70 Reducing the time required to 
review and audit rejected credit returns 
and rebate forms would reduce the amount 
of interest incurred on refund claims paid 
after 21 days. It would also improve 
service to GST registrants. 
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Tax services offices 
are having trouble 
keeping up with the 
workload generated by 
the automated validity 
checks. 


16-20 


16.71 We noted that tax services offices 
are having trouble keeping up with the 
workload generated by the automated 
validity checks — both at the initial 
review stage and later, when the work is 
assigned to auditors. In 1997-98, 
according to Revenue Canada officials, 

72 percent of the pre-payment audits took 
longer than 30 days to complete and. 
accounted for 13 percent of the amounts 
paid to registrants. In a sample of 27 
pre-payment audits that we selected from 
among the five tax services offices that we 
visited, we noted delays in completing 17 
of them. Nine took more than 180 days to 
complete. We determined that the audit 
delays were caused primarily by the 
registrant in 7 of the 17 cases and in 

10 cases by Revenue Canada. 


16.72 Delays caused by the registrant 
consisted primarily of the time it took the 
registrant to provide requested 
information. Delays caused by Revenue 
Canada consisted primarily of the interval 
between rejection of the refund claim by 
an automated validity check and 
commencement of the auditor’s work. Tax 
services officials informed us that these 
delays were due mainly to the workload 
created by the large volume of refund 
claims. 


16.73 If a pre-payment audit were to 
ultimately prevent a refund cheque from 
being issued, the fact that it had taken 
many months to complete would not add 
to Revenue Canada’s interest costs. 
However, of the 17 delayed audits in our 
sample, 15 ultimately resulted in a cheque 
being issued to the registrant. In total, the 
registrants’ claims were reduced by 

38 percent. While this is more than the 
17 percent reduction for all 1997-98 
credit returns (see Exhibit 16.6), it means 
that the registrants received interest for 
the period of the delay on 62 percent of 
the amount claimed. 


There is a need for up-to-date 
procedures to review refund claims 


16.74 We noted that the Department’s 
national procedures for reviewing credit 
returns and rebate forms are not 
up-to-date. As a result, they are not 
always being followed. Some tax services 
offices that we visited have published 
their own procedures, while other offices 
have left the matter up to each individual. 
There is a risk that some reviewers are 
performing too little work, thus failing to 
identify inappropriate refund claims, 
while others are performing too much 
work — an inefficient use of audit time. 


16.75 Staffing matters are compounding 
this risk. Officials at three of the five tax 
services offices that we visited informed 
us that the units responsible for reviewing 
the refund claims were experiencing high 
staff turnover. At two offices, we were 
informed that there was a move toward 
using lower-level staff to review the 
claims. If this were to occur, decisions 
could be made by staff who lack sufficient 
knowledge of the office’s registrant 
population to be able to identify the 
highest-risk credit returns and rebate 
forms. Again, the reviewer might perform 
either too little or too much work on a 
particular item. 


16.76 Revenue Canada should 
intensify its review of the procedures 
used in the tax services offices to review 
and audit refund claims. This would 
include reviewing how the offices 
control their workload, and the number, 
levels and experience of audit staff 
performing the various functions. 


Department’s response: The Department 
is in agreement with this recommendation 
and has already completed some work in 
this regard, and has initiated further study 
and review activities as suggested by the 
Auditor General. 


e We have reviewed the procedures 
used in the tax services offices (TSOs) to 
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review and audit refund claims and 
obtained from them lists of best practices. 
This information is being included in the 
updated guidelines that are being 
developed within the Verification, 
Enforcement and Compliance Research 
(VECR) Operational Manuals project. 
These guidelines will include updated 
procedures as well as review and audit 
techniques to process and select credit 
returns for examination, and updated 
procedures for setting individual and 
cumulative thresholds. Increased 
monitoring will be performed to ensure 
that thresholds are appropriately set. 


e Asa result of a focus group meeting 
held in May 1999 to examine ways to 
improve overall program delivery for all 
Small and Medium Enterprise programs, 
guidelines will be issued shortly to all 
TSOs recommending that the initial review 
and evaluation of credit returns be 
consolidated within the Verification and 
Enforcement area of the TSO responsible 
for workload selection. This will provide 
for greater consistency in the way “tax at 
risk” is identified and further ensure that 
experienced staff address the review of 
credit returns. Many TSOs have already 
moved to this type of approach. 


e The VECR Branch recently 
concluded a study relating to team size 
and will be recommending that TSOs move 
to smaller teams, which will provide for 
closer monitoring of the workload by the 
first line supervisor. 


GST Post-Payment Audit Program 


16.77 The post-payment audit program 
is a tax services office function. Teams of 
post-payment auditors select registrants 
for examination. They generally select the 
registrants they believe represent the 
highest risk of non-compliance. The file is 
then assigned to an auditor who plans the 
audit, performs the work and, if 
appropriate, issues an assessment. In this 
section, we examine each of the three 
main steps — audit selection, audit 
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planning and audit performance. Some of 
our findings are based on a sample of 
141 post-payment audits that we selected 
from among five tax services offices. 


Better data and tools would improve 
Revenue Canada’s audit selection 
process 


16.78 Asa first step in assessing the 
risk of non-compliance, Revenue Canada 
groups its registrants into two general 
categories, based on their annual revenue. 
“Large files” — those representing the 
highest revenue — are supposed to have 
all tax years audited in full. (However, 
owing to staff shortages, not all tax years 
are being audited in full in the tax services 
offices that we visited.) “Small and 
medium enterprises” — those with lower 
revenues — are selected for audit using a 
risk-based approach, taking into account 
such factors as the results of any previous 
audits, the industry sector, and how 
closely the GST they have reported 
matches industry norms. Small and 
medium enterprises can be selected for 
audit through numerous means, including: 


e external leads from third parties; 


e internal leads or referrals from other 
units in the tax services office or 
elsewhere in Revenue Canada; 


e Revenue Canada’s Computer 
Assisted Audit Selection system; and 


e judgment based on sources such as 
industry directories and business listings 
in the yellow pages of telephone 
directories. 


16.79 Not all of these selection methods 
are equally effective at finding 
non-compliance. Some may work better in 
some industry sectors or geographic 
regions than others. 


16.80 Revenue Canada cannot tell the 
extent to which the various audit 
selection methods are being used. To 
help determine the most effective 
selection methods to use for a particular 
sector or in a particular region, Revenue 


Returns Processing and Audit 


The Department does 
not have detailed 
information on how 
registrants have been 
selected for 
post-payment audit. 
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There are deficiencies 
in Revenue Canada’s 
Computer Assisted 


Canada needs to know how well various 
methods have worked in the past. 
However, the Department does not have 
detailed information on how registrants 
have been selected for post-payment audit. 
The audit selection codes currently 
captured in the GST mainframe and in the 
Audit Laptop System are not recorded 
consistently and are sometimes not 
specific enough to identify the method of 
selection. Furthermore, during the period 
covered by our audit, not all offices were 
using the selection codes set up in the 
Department’s Audit Information 
Management System (AIMS) to 
complement the Audit Laptop System. We 
understand that the tax services offices 
have since begun using the audit selection 
codes in AIMS. 


16.81 One tax services office that we 
visited had routinely used narrative to 
complement the audit selection codes, so 
we were generally able to determine the 
selection method. In the other four offices, 
we were unable to determine the selection 
method for over 40 percent of our sample 
items. 


16.82 The effectiveness of one of 
Revenue Canada’s audit selection 
methods is hampered by some data 
deficiencies and a need for ongoing 
training. When Revenue Canada’s 
Computer Assisted Audit Selection 
(CAAS) system is combined with the 
Audit Report Generator On-line (ARGO), 
the system allows auditors to focus on 
specific industry sectors, geographic 
locations and audit issues. The system 
contains GST and income tax data, some 
provincial sales tax data, and data on 
importers. Auditors can use this 
information to identify registrants who do 
not fit the usual profile of the industry, or 
whose GST and income tax data appear to 


While we consider CAAS/ARGO 
to have potential for selecting registrants 
with a high risk of non-compliance, 
Revenue Canada auditors had used it to 


Audit Selection 

system. 
be inconsistent. 
16.83 

L6=22 


select only 7 percent of the post-payment 
audits in our sample. Revenue Canada 
officials at headquarters and some auditors 
tasked with using the tool cited certain 
deficiencies in it. For example, 
information on gross revenues and other 
pertinent business data for each registrant 
are not up-to-date. Also, there are no data 
on export sales. (As these sales are not 
taxed, they can significantly distort ratios 
used to highlight potential 
non-compliance.) Revenue Canada 
officials identified these and other 
deficiencies as early as 1996 but, despite 
their efforts, have not yet been able to 
resolve them. At present, they are 
considering adding the reporting of export 
sales to the GST returns. 


16.84 The effectiveness of 
CAAS/ARGO is also hampered by 
missing or incorrect standard industry 
classification codes used to group 
registrants by industry sector. The use of 
CAAS/ARGO to select registrants in a 
particular industry sector would not 
capture registrants with missing or 
incorrect codes. Despite recent 
improvements, in May 1999 over 

10 percent of the registrants in Revenue 
Canada’s database still did not have 
standard industry classification codes. In 
our post-payment audit sample, 9 percent 
did not have codes and, in our view, more 
than 10 percent of the other registrants 
had codes that were incorrect. 


16.85 Further, based on interviews 
conducted in the five tax services offices 
that we visited, it appears that the 
effectiveness of CAAS/ARGO is 
hampered by the lack of training received 
by some employees tasked with using the 
system. Revenue Canada officials 
informed us that high staff turnover may 
be contributing to this. 


16.86 Revenue Canada should 
continue to improve its Computer 
Assisted Audit Selection and Audit 
Report Generator On-line systems. This 
would include dealing with the 
identified deficiencies, ensuring that all 
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necessary data (such as standard 
industry classification codes) are 
available, and providing additional 
training to staff who are expected to use 
the system. 


Department’s response: Although 
Revenue Canada’s Computer Assisted 
Audit Selection and Audit Report 
Generator On-line systems for GST are 
relatively new, we have been seeking and 
will continue to seek ways to improve 
them. This ongoing process includes 
incorporating new data from various 
sources and increasing the completeness 
and accuracy of currently available data. 


We recognize the importance of data on 
export sales and will continue our efforts 
to obtain such data. We also recognize the 
importance of standard industry 
classification codes and are working with 
Officials from Statistics Canada to address 
this. 


Although we have been providing ongoing 
training on the use of these systems, we 
recognize the need for continuous training 
and will increase our efforts in this area. 


16.87. Random audits are needed. One 
way to identify the sectors with the 
highest rates of non-compliance would be 
to periodically select a random sample of 
registrants in which all sectors have an 
equal chance of being represented. This 
sample could also be used to measure 
changes in levels of non-compliance over 
time. After many delays, Revenue Canada 
has a project under way to select a random 
sample from one industry sector. Samples 
were sent to the tax services offices in 
February 1999, and the first audit results 
are due in December 1999. 


More attention needs to be given to 
audit planning 


16.88 Once a registrant has been 
selected for audit, the file is assigned to an 
auditor. The auditor is then responsible for 
developing an audit plan, performing the 
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work and, if appropriate, issuing an 
assessment. 


16.89 While good selection techniques 
will help to ensure the selection of 
registrants with the highest risk of 
non-compliance, they must be followed by 
good planning to ensure the efficient and 
effective use of audit resources. 


16.90 Documentation of planning is 
limited. To assist in the planning process, 
Revenue Canada’s Audit Laptop System 
contains suggested planning procedures. 
The procedures are not designed to deal 
with many industry sectors or complex 
issues. Given these limitations, Revenue 
Canada requires its large file auditors to 
augment the planning procedures in the 
Audit Laptop System with written audit 
plans. However, of the eight large file 
audits that we believe needed an audit 
plan, only four had a written plan. 


16.91 Revenue Canada officials have 
informed us that the Department also 
requires its auditors of small and medium 
enterprises to augment the planning 
procedures in the Audit Laptop System 
with written audit plans. However, most of 
the audit files in our sample did not 
contain these plans. 


Most files in our 
sample did not contain 
audit plans. 


16.92 Revenue Canada’s Quality 
Assurance Division also concluded that 
more documentation of planning would be 
beneficial. It also noted that better training 
— both through courses and on the job — 
would help to improve the planning 
process. 


There are areas of strength in audit 
performance 


16.93. Once the audit has been planned, 
the auditor performs the work. This 
includes completing the work described in 
the plan and reaching appropriate 
decisions as to which issues should be 
pursued and which should be dropped. It 
also includes documenting, either in the 
Audit Laptop System or in hard copy 
working papers, the work performed and 
the key decisions made. Exhibit 16.7 
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Thirty-two percent of 
the audits had been 
under way for more 
than 180 days; these 
accounted for 

70 percent of the 
cumulative GST 
audit hours. 


Exhibit 16.7 


Example of a Good 
Post-Payment Audit 


contains an example of what we consider 
to be a good post-payment audit. 


16.94 We noted areas where all tax 
services offices were performing well. For 
example, on most audits: 


e there was evidence in the working 
paper files (Audit Laptop System or hard 
copy) that auditors had taken into account 
previous experience with the registrant 
when determining which issues to audit; 


e there was no evidence in the files 
that issues raised were not adequately 
addressed; and 


e the files contained support for the 
amounts being assessed. 


16.95 We also noted areas where some 
tax services offices were performing well. 
For example, two offices had expanded 
their audits of registrants to incorporate 
audits of related parties, in the majority of 
the cases where this work appeared to be 
appropriate. Auditing related parties 
concurrently can determine whether 
transactions between the parties were 
handled correctly for GST purposes. 


16.96 We also noted that some tax 
services offices have enhanced the 
standard audit support tools. For example: 


e one office has developed a software 
application to facilitate monitoring of the 


work in progress and a standard audit 
program to be used on its smallest 
registrants; 


e another office has developed 
guidelines for dealing with requests under 
Revenue Canada’s audit “fairness 
package” and a course dealing with net 
worth audits and reasonability tests; and 


e athird office has developed a 
comprehensive set of audit programs and 
a working paper index for large file audits. 


Some aspects of audit performance 
require improvement 


16.97 Our sample of post-payment 
audit files revealed some areas in need of 
improvement. Many audits took many 
months and many hours to complete. 
There was usually no evidence of active 
participation by the team leaders. Some 
policies were not followed and some 
procedures were not performed. These are 
discussed below. In addition, the audit 
described in Exhibit 16.8 provides 
examples of needed improvements. 


16.98 Many audits took many months 
to complete. A national report of GST 
audits in progress on 22 March 1999 
showed that 32 percent of the audits had 
been under way for more than 180 days. 
These accounted for 70 percent of the 
cumulative GST audit hours. This means 
that audits begun in the past six months 


The auditor began by comparing GST revenue, as reported by the registrant on its GST returns, 
with the sales amount reported in the registrant’s financial statements. The auditor noted an 
unexpected decline in GST revenue, and adjusted his audit program to deal with this. This work 
revealed clerical errors and sales invoices on which GST had not been charged correctly. These 
errors totalled approximately $90,000. ; 


The auditor also checked for clerical errors in the input tax credit account. He discovered that the 
registrant was entitled to input tax credits in respect of purchases on which GST of approximately 
$50,000 had been paid. 


A review of other selected general ledger accounts and bank statements did not reveal any further 
anomalies, so the auditor did not perform any more detailed tests. The audit work was completed 
in approximately 100 hours, and less than two months elapsed between the file’s assignment to the 
auditor and the issuance of the notice of assessment. The audit working papers contained a 
complete index, a section that provided an overview of the audit and a summary audit program, 
and schedules that clearly supported all amounts being assessed. An income tax compliance 
checklist had also been completed. 
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accounted for only 30 percent of the audit 
hours. 


16.99 — In our own audit sample, 67 of 
the 141 post-payment audits had taken 
more than 180 days, and 14 had lasted 
more than two years. The average 
duration of the 67 audits was 495 days — 
about 16 months. 


16.100 The extensive length of these 
audits was the result of delays caused by 
either Revenue Canada or the registrant or 
both. Delays by Revenue Canada were 
due primarily to conflicting priorities of 
the auditors involved. Delays caused by 
registrants were primarily due to a failure 
to respond to requests for an initial 
meeting or to provide requested 
documentation on a timely basis. 


16.101 Some of the delays caused by 
Revenue Canada could have been avoided 
by better audit planning and increased 
monitoring of the work in progress. In one 
tax services office, we were told that 
audits had not been monitored because 
team leaders were being rotated every few 
months. Detailed monitoring was 
performed only after permanent team 
leaders had been assigned. Another office 
told us that for various reasons there had 
been no detailed monitoring, so many 
audit files had been outstanding for more 
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than 180 days. We were also informed that 
to make the workload more manageable, a 
number of old files had been reviewed and 
a decision made to discontinue work on 
those expected to generate low or no 
assessments. 


16.102 Registrant delays can add 
significantly to the cost of the audit, and 
are often beyond Revenue Canada’s 
control. It can penalize registrants who do 
not provide requested data promptly. 
While we understand that the Department 
does impose these penalties, and some 
files in our audit sample indicated that it 
had threatened to impose them, it had not 
done so in any of our sample items. 


16.103 Some GST audits were not 
efficient. Audits that take many hours but 
yield only small assessments are not 
necessarily poor audits. Very large, 
complex registrants may take hundreds of 
hours to audit, and may turn out to be in 
compliance with the Excise Tax Act. On 
the other hand, a pattern of such audits 
could be a symptom that they were poorly 
done. 


16.104 To determine whether audits that 
take many hours but yield only small 
assessments are good or poor audits, we 
reviewed the 25 audits in our sample that 
had taken more than 100 hours and had 
yielded a revenue recovery of less than 


The audit file was assigned to the auditor in November 1992. The auditor completed his field work 
and sent a proposed assessment to the registrant in May 1995. For various reasons, the notice of 
assessment was not issued until December 1997 — more than 18 months after the proposed 
assessment and more than five years after the assignment date. Almost 800 hours had been charged 
to the audit. The assessment was for about $31,000. Penalty and interest had been reduced due to 
the excessive time between the completion of the fieldwork and the issuance of the assessment. 


Most of the assessment dealt with inappropriate input tax credits on employee automobile benefits. 
Other miscellaneous matters were assessed, including $200 of unreported GST on sales to 
employees. In our opinion, the extent of testing and documentation to support the amounts 


assessed appeared excessive. 


During the five-year period, there was a change in the team leader assigned to the audit. There was 
no evidence that either team leader had been actively involved in the audit. 


The working papers did not indicate how or why the registrant had been selected for audit. There 
was no evidence that any reasonability tests had been performed. 


The audit had been recorded on the GST mainframe as a large file audit; this was not the case. 


Returns Processing and Audit 
~~ 


Registrant delays can 
add significantly to the 
cost of the audit. 


Exhibit 16.8 


Example of a Post-Payment 
Audit Requiring Improvement 
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Team leader 
involvement in audits 
is limited. 


$100 per hour. We found that in over 

75 percent of the cases, the time spent 
appeared to be excessive in relation to the 
size and complexity of the registrant and 
the scope of the audit work. This not only 
represents an inefficient use of resources 
but also adds to the burden on the 
registrants being audited. 


16.105 Revenue Canada officials have 
informed us that post-payment audits that 
had taken more than 100 hours and 
yielded a revenue recovery of less than 
$100 per hour constituted only 5 percent 
of the audits completed in 1997-98. 
Nevertheless, we believe that excessive 
audit time is a matter that Revenue 
Canada needs to pursue. We found in 
sample items other than the 25 just noted 
that audit time had been excessive in 
relation to the size and complexity of the 
registrant. In addition, Revenue Canada’s 
Quality Assurance Division also reported 
that it found many audits conducted in 
1996 and 1997 that had taken an excessive 
length of time in relation to the audit 
scope, and that many audits went on far 
too long. The report concluded that better 
audit planning and greater management 
participation would reduce the time spent 
on audits. 


16.106 One way to control the number of 
hours spent on an audit would be to 
budget at the planning stage for a certain 
number of hours. Ideally, budgets would 
be tailored to each set of circumstances to 
provide a better basis for managing and 
controlling audit performance. This is not 
being done. Instead, tax services offices 
often use national averages, developed by 
Revenue Canada as part of its annual 
budgetary exercise. These national 
averages fail to account for the unique 
characteristics of each audit. One tax 
services office that we visited now 
requires staff to sign “performance 
communications”, stating that, among 
other things, they will normally complete 
their audits in less time than the national 
averages. While we understand that these 
documents were implemented to provide 


the office’s management with greater 
control over the number of hours spent on 
each audit — a commendable goal — we 
are concerned that this approach could 
lead staff to unduly limit the scope of their 
audits. 


16.107 Another way to control the time 
spent on an audit would be to increase 
active participation by team leaders. This 
would also help to ensure that audits do 
not drag on too long and are performed as 
planned. Seventy percent of our 
post-payment audit files showed no 
evidence of such participation. 


16.108 Ina recent report, Revenue 
Canada’s Quality Assurance Division also 
reported that team leader involvement in 
audits is very limited. The report cites 
various contributing factors, such as group 
size and the heavy demands of other 
responsibilities. The Department is 
considering reducing the size of groups 
and redefining the roles and 
responsibilities of team leaders. 


16.109 Revenue Canada should 
enhance its efforts to reduce the 
duration of its audits and the hours 
required to complete them. These 
efforts should include enabling team 
leaders to be more actively involved in 
their audits, and providing greater 
support to auditors in planning and 
performing their work. Efforts should 
also include making more use of 
penalties to deter delays by registrants. 


Department’s response: As described 
below, Revenue Canada has already taken 
action to improve overall program 
delivery in the areas identified in this 
recommendation. 


e In the budget exercise for 
1999-2000, very specific accountabilities 
have been put in place to ensure the 
achievement of overall program 
objectives. The tax services offices (TSOs), 
through the regional offices, will report to 
headquarters starting with the six months 
ending September 1999 on the progress 
achieved in relation to meeting agreed-to 
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objectives. The report will include 
comments on the reasons for any 
variances and the action plan to address 
the variances. This will include status 
reports on files outstanding for more than 
six months. 


e The Department has completed a 
Team Size study, which recommends a 
reduction in team sizes. This will be 
discussed with regional and tax services 
office management with a view to formal 
implementation in 2000-2001. 
Implementing the recommendations from 
this study will enable the team leaders to 
spend more time with the auditors and 
become more actively involved in the 
audits. Most TSOs have already 
implemented the smaller team size 
structure recommended in this study. 


e Incollaboration with the Small 
Business Advisory Group, the Department 
developed an audit pamphlet, which will 
be provided to all taxpayers/registrants 
when an audit is started. The pamphlet 
clearly sets out the responsibilities of the 
taxpayer/registrant and the auditor in the 
audit process. With everyone having a 
clear understanding of their 
responsibilities, there will be fewer delays 
on audits. 


e In May 1999 the Department held a 
focus group session involving staff from 
TSOs to develop an action plan to improve 
overall program delivery. The 
recommendations from this focus group 
are being implemented this fiscal year. 
One of the recommendations coming out 
of this focus group is to make more use of 
formal requirements where 
taxpayers/registrants or tax practitioners 
are not providing information within 
reasonable time frames. Prosecution 
action will be initiated where there is 
non-compliance with the requirements. In 
addition, the Department will look at the 
possibility of increased use of civil 
penalties for failure to provide 
information. 
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16.110 The audit period often exceeded 
Revenue Canada’s policy without 
justification. One of Revenue Canada’s 
policies is to restrict the coverage of each 
GST audit to the two preceding tax years, 
along with returns filed during the current 
year. Earlier years are to be audited only 
in limited circumstances, and only on the 
approval of the team leader. This policy is 
designed to improve audit efficiency and 
reduce the burden on registrants. 


16.111 We noted that the audit period for 
76 of our 141 post-payment audits 
exceeded the period called for in the 
policy, in a large number of cases without 
justification. The Department has issued a 
directive to its auditors of small and 
medium enterprises to reinforce the 


policy. 


16.112 Incorporating a compliance 
review of income tax into GST audits is 
not a common practice. The strategy for 
audits of small and medium enterprises 
calls for the integration of income tax and 
GST audits to the extent possible. In cases 
where only one tax is audited, a 
“compliance review” of the other tax is 
generally required. This work entails 
examining specific indicators of the 
second tax to determine the level of 
compliance. Compliance reviews offer 
increased coverage, which should lead to 
greater deterrence and increased revenue 
recovery. 


In cases where only 
one tax is audited, a 
“compliance review” 
of the other tax is 
generally required. 


16.113 Using the Department’s own 
criteria, we concluded that an income tax 
compliance review checklist had been 
warranted in 77 of the post-payment audit 
files we reviewed. Fifty-two of these files 
contained no evidence that these 
checklists had been completed. 
Departmental officials have informed us 
that they recently made changes to the 
Audit Information Management System to 
provide management with the information 
needed to ensure that compliance reviews 
are done when necessary. 
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Auditors often do not 
perceive that they 
have a promising 
career path in the 
GST stream. 


16-28 


Support for the GST Audit 
Programs 


Staffing and training needs 


16.114 The success of the pre-payment 
and post-payment audit programs depends 
on the availability of enough qualified, 
well-trained and well-equipped staff. The 
ideal number and mix of staff in terms of 
knowledge, skills and experience varies 
among tax services offices, depending on 
the characteristics of the registrant 
population served by each office. 


16.115 Senior managers in all five tax 
services offices that we visited stated that 
they do not have the ideal complement of 
audit staff and equipment. Two offices 
have a surplus of auditors at the AU-1 
level (who typically audit smaller 
corporations) and a shortage of “generalist 
auditors” at the PM—2 level (who typically 
do both GST and income tax audits of 
small unincorporated businesses). Three 
offices have a shortage of senior auditors 
(AU-3 level) who audit large 
corporations. 


16.116 Senior managers and auditors in 
the five tax services offices informed us 
that the shortage of senior auditors 1s 
made worse by high turnover, due to staff 
transfers within the Department as well as 
staff departures. Auditors often do not 
perceive that they have a promising career 
path in the GST stream, and prefer to 
switch to the income tax stream. This 
perception has its roots in differences in 
the job descriptions of auditors, predating 
the consolidation of the former 
departments of Taxation (which 
administered the income tax) and Customs 
and Excise (which administered the GST). 
For example, the job descriptions call for 
GST auditors at the AU—2 level to 
perform audits of the larger small and 
medium enterprises, while the income tax 
audits of these enterprises are done by 
more senior auditors, at the AU-—3 level. 


16.117 As for training, the generalist 
auditors at the PM—2 level in the five tax 
services offices claimed that their training 
was excellent. Other GST auditors 
informed us that they lacked training in 
specialized areas. In particular, some 
auditors told us that they felt ill-prepared 
to audit complex and specialized 
organizations, such as public sector bodies 
(for example, municipalities). In some 
offices, technical advisors have 
augmented the official training program. 
When time permits, these experts in 
legislative interpretation give 
presentations on technical topics of local 
interest. 


16.118 Departmental officials have 
informed us that they have commenced a 
number of initiatives to attract and retain 
knowledgeable staff. 


Information needs 


16.119 To improve its GST pre-payment 
and post-payment audit programs, 
Revenue Canada needs to know which 
activities are working well and which ones 
need improvement. This means that the 
Department needs consistent and reliable 
information that can be aggregated and 
compared with GST assessments. 


16.120 Problems in the consistency of 
the data. In previous reports, we have 
commented on problems of data 
consistency. In this audit, we again 
encountered inconsistent data. Until 
Revenue Canada can assure itself that the 
data are consistent, it will not know which 
variances in audit results — among tax 
services offices and year to year — are 
attributable to errors and inconsistencies 
and which are attributable to “real” 
performance factors that should be 
followed up. These data problems will 
limit Revenue Canada’s ability to improve 
its pre-payment and post-payment audit 
programs. 

16.121 We noted the following 


problems: 


e inconsistencies in recording the time 
spent initially to review the 
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audit-worthiness of a particular return or 
registrant; 

e inconsistencies in recording audit 
types; and 


e recording different audit times in the 
Audit Laptop System and the Audit 
Information Management System in four 
of the five offices we visited. (In the fifth 
office, differences were being reconciled.) 


16.122 Some necessary data are not 
being captured, or are not being 
captured in ways that can easily be 
aggregated. Revenue Canada does not 
have aggregated data on the kinds of 
non-compliance being identified during 
post-payment audits, or on the scope of its 
post-payment audits. (These may be full 
audits or restricted to specific issues.) If 
these data could be aggregated and 
matched in aggregate to resulting 
assessments, Revenue Canada would be 
better able to determine the most 
prevalent forms of non-compliance and 
the effectiveness and efficiency of its 
audits. 


16.123 Some aggregated data are not 
readily available. In order to make good 
audit selections, management needs 
information such as the number of 
registrants in each industry sector or in 
various ranges of sales revenue, prior 
audits of each registrant, audit hours and 
amounts assessed by industry sector, and 
adjustments to assessments made by the 
Appeals Branch. Although this 
information exists in various databases 
and reports, it is sometimes difficult to 
access and analyze. In these cases, 
management must rely on special ad hoc 
reports and registrant tape files that are 
updated and distributed every six months. 


16.124 Revenue Canada should 
continue to improve the timely 
availability of data needed to manage 
the GST audit programs effectively. To 
that end, the Department should resolve 
inconsistencies in the data-capturing 
methods, ensure that missing data are 


Report of the Auditor General of Canada — September 1999 


Revenue Canada — Goods and Services Tax: 
Returns Processing and Audit 


obtained, and ensure that data can be 
aggregated and analyzed. 


Department’s response: The Department 
agrees with this recommendation; some 
work has been completed and other work 
is under way to address these issues. 


e GST audits were included in the 
Audit Information Management System 
(AIMS) to enable managers in the tax 
services offices and in the regions to 
better manage program delivery. The 
Department will continue to modify AIMS 
to provide more information on GST 
audits, such as the reasons for audit 
selection. 


e The Department is using 
state-of-the-art technology to develop 
applications (GST Power Play Cubes 
using Impromptu), which will enable all 
levels of management to more easily 
aggregate and analyze information on 
GST audits completed and in process. The 
Department is also developing 
applications for managers using 
Electronic Business Intelligence System 
software that will enable all levels of 
management to more easily benchmark 
progress in relation to key performance 
indicators. These initiatives will facilitate 
increased effectiveness in the management 
of the GST audit program. 


Revenue Canada does 
not have aggregated 
data on the kinds of 
non-compliance being 
identified. 


e The Department is in the early stages 
of developing a comprehensive 
Management Information System to meet 
the information needs of all programs 
within the Verification, Enforcement and 
Compliance Research Branch and to fully 
implement the Management Information 
Agreements that have been put in place. 
This system will provide considerable data 
relating to program delivery and also 
more information on the types and dollar 
values of non-compliance identified by 
audits. 


e The Department is also developing a 
Corporate Performance Measurement 
Framework based on the Balanced 
Scorecard methodology for all programs. 
Building this system will entail the 
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The Department is in 
the early stages of 
developing a 
comprehensive 
management 
information system. 


redesign of the Department's entire 
measurement system, from the 


development of corporate measures to how 


measures are used and interpreted by 
management and employees. 


16.125 Revenue Canada makes few 
comparisons with other revenue 
administrations. Comparing Revenue 
Canada’s audit selection, planning, 
performance and results with those of 
similar organizations could help it to 
measure current success and identify best 
practices, thus leading to improvements. 


16.126 While the ministére du Revenu 
du Québec uses all of Revenue Canada’s 
systems and policies, it sets its own audit 
strategy. Revenue Canada officials 
informed us that, while they maintain 
open communication with the ministére 
on operational and administrative matters, 
they do not verify the completeness or 
accuracy of the data received from the 
ministére or perform detailed analyses of 
its audit results. Revenue Canada is close 
to finalizing an updated agreement with 
the ministére that provides for a more 
structured sharing of audit strategies, work 
plans and audit results. 


16.127 As for other organizations, 
Revenue Canada officials informed us that 
they have ongoing dialogue and attend 
international gatherings with similar 
organizations in other countries with GST 
(value-added tax); however, they have not 
formally compared their compliance 
efforts with those countries. 


Control over audit files 


16.128 Revenue Canada needs to be able 
to locate its audit files when registrants 
appeal their assessments. However, for 8 
of the 168 pre-payment and post-payment 
audits in our sample, the hard copy 
working papers could not be located. 
Similarly, for audits of 17 registrants, the 
Audit Laptop System diskettes either 
could not be found or were unusable. 


16.129 The primary reason given for the 
lost and misplaced hard copy files was the 
absence of an electronic control log to 
track who in Revenue Canada has the 
GST audit files when they are transferred 
among various units — for example, 
Appeals Branch, Special Investigations, 
headquarters or other tax services offices. 
The primary explanation for the unusable 
diskettes was that they malfunctioned 
during the audit and the audit working 
papers contained therein could not be 
retrieved. Departmental officials have 
informed us that they are currently 
addressing these matters. 


Reporting on Performance 


16.130 Under its governing legislation, 
the new Canada Customs and Revenue 
Agency will be required to establish and 
report on performance in relation to its 
objectives and performance expectations. 
We found that production data for the 
GST processing function are collected and 
made available in a relatively timely 
manner. However, we found very few 
instances of these data being used to 
arrive at accurate performance measures. 
As for audit, the Department is in the 
early stages of developing a 
comprehensive management information 
system to meet the needs of its 
Verification, Enforcement and 
Compliance Research Branch, which is 
responsible for the GST audit function. It 
is important that the Department correct 
the deficiencies we have noted throughout 
this chapter with respect to the 
availability, consistency and reliability of 
data and that it complete the information 
system it is developing. Otherwise, the 
new Agency’s ability to report on 
performance as required by its Act may be 
jeopardized. 


Conclusion 


16.131 Returns processing and audit are 
two core functions of GST administration. 
Our audit of these functions revealed both 
examples of good performance and areas 
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for improvement. In the audit function 
particularly, we noted that Revenue 
Canada has identified several areas for 
improvement and has launched 
improvement initiatives, but the results of 
these measures are not yet known. 


16.132 The timeliness with which 
documents are processed at the 
Summerside Tax Centre has improved 
since the Centre opened in 1993. Our 
examination of the timeliness of 
post-payment audits revealed that many 
audits have extended over many months 
and have taken many hours to complete. 
Pre-payment audits by their nature do not 
last as long as post-payment audits, but 
these, too, are prone to delays. Both 
Revenue Canada and the registrants 
themselves have contributed to delays in 
the completion of audits. We believe that 
further improvements are possible in the 
speed with which documents are 
processed and that additional steps can be 
taken to reduce the delays experienced in 
audits. This would result in better service 
to registrants and, in some situations, 
reduced interest charges on refunds paid 
after the 21-day grace period. Document 
processing, pre-payment audit and 
post-payment audit activities are linked, 
and the persistence of problems with 
timeliness suggests that performance in 
this network of activities has not yet been 
optimized. 


16.133 Revenue Canada has made 
progress in establishing performance 
standards and expectations for its returns 
processing and audit activities, and is 
undertaking a major overhaul of its 
management information systems. At 
present, however, standards for returns 


processing tend to place insufficient 
emphasis on quality, accuracy, timeliness 
and unit cost, and the use of performance 
information is hampered by problems with 
the availability and reliability of data. In 
the Department’s audit activities, the use 
of performance information is hampered 
by problems with the availability and 
consistency of data. 


16.134 The Department faces a 
significant challenge in maintaining a 
sufficient complement of GST staff with 
appropriate knowledge, skill and 
experience. Staff transfers within the 
Department and staff departures contribute 
to shortages of staff in particular 
activities. Training for staff who must 
perform new functions has not always 
kept up with the need. 


16.135 Prompted by the decision to 
decentralize the processing of GST 
returns, formerly performed only at the 
Summerside Tax Centre, Revenue Canada 
recently simplified its procedures. This 
follows efficiency gains made over the 
years in Summerside. Revenue Canada 
has also improved the automated validity 
checks it uses to select, for possible 
pre-payment audit, credit returns with the 
highest risk of non-compliance. However, 
we believe that further improvements are 
possible to make the checks more 
effective, and that the Department should 
consider reviewing and auditing debit 
returns at the same time as it conducts 
pre-payment audits of credit returns. Also, 
the performance of both pre-payment and 
post-payment audits could be improved by 
the consistent application of existing or 
strengthened procedures. 


Our audit revealed 


both examples of good 
performance and areas 


for improvement. 
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About the Audit 


4 


Objective and Scope 


The objective of the audit was to assess the efficiency and effectiveness of Revenue Canada’s GST returns 
processing and audit activities. 


The audit took place at Revenue Canada headquarters in Ottawa and at various tax centres and tax services 
offices throughout the country. 


Criteria 
We expected that: 


e processing and audit of GST returns, and payment of refunds and rebates, would be done on a timely 
basis; 


e management would work continually to optimize the performance of the entire network of GST functions 
and activities; 


e performance expectations for GST returns processing and audit activities would be set out in 
departmental plans. Management information systems would track related performance information; 


e the Department would have recruited, trained, developed and maintained a sufficient complement of staff 
with appropriate knowledge, skill, and experience; 


@ operations would be designed and carried out using efficient systems, processes, procedures and work 
methods. The performance of risk assessment criteria and audit methodology would be regularly 
monitored, assessed and updated; and 


e adequate information systems would be in place that ensure the reliability, consistency, availability and 
security of key information used in GST processing and audit. 


Audit Team 


Assistant Auditor General: Shahid Minto 
Principals: Scott Milne and Jim Ralston 
Directors: John Pritchard and Richard Quesnel 


Terence Brown 
Thérése Desjardins 
Nisha Goyal 
Patricia Smith 


For information, please contact Scott Milne. 
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Chapter 17 


Canada Infrastructure 
Works Program 


Phase II and Follow-up 
of Phase | Audit 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 
Accountants. The numbered paragraphs in bold face represent recommendations. 


Table of Contents 


Main Points 


Introduction 


Objectives and structure of the Program 


The allocation of responsibilities between the federal and provincial governments is based 


on their comparative expertise 
Focus of the audit 


Observations and Recommendations 


Program Implementation 


Some program differences exist between Phase I and Phase II 
Both phases of the Program were implemented quickly 
Program officials view the program approach very positively 


Effort was devoted to ensuring that participation of all partners was recognized 


Funding, Targeting and Impacts of the Program 


Funds were spent largely on traditional infrastructure 
Program targeting is weak 
The implications of private-sector-related projects need to be addressed 


Incentives exist to substitute Canada Infrastructure Works Program funds for regular, 


ongoing provincial infrastructure expenditures 
Safeguards are needed to limit program substitution 
Continuing questions about job creation estimates 


Financial and Management Controls 


Insufficient attention to ensuring control 

Impiementation of compliance audits has been slow 

Insufficient assurance of value for money 

Improvements made in fulfilling federal environmental obligations 


Response to 1996 Audit 


Limited progress on the recommendations of the 1996 Phase I audit 
Improvements are needed in federal-provincial agreements 


Conclusion 

About the Audit 

Exhibits 

17.1 Federal Allocation of Funds Under Phase II 

Le Total Expected Investments by Type of Infrastructure: Phase I and Phase II 

17.3 Frequency of Program Criteria Selected and of Claims Supported by 
Documentation Under Phase I] 

17.4 Phase II Projects Involving Private Sector Partnerships and Other 
Private Sector Linkages 

GES Local Infrastructure Investment Assistance Provincial Government 
Cost-Share Ratio (%) Under Canada Infrastructure Works Program and 
Other Selected Provincial Capital Assistance to Municipalities — 1993-1999 

17.6 Compliance Procedures Used Under Phase II 

Mes Assessment of Program Response to the 1996 Report of the Auditor General 
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Canada Infrastructure 
Works Program 


Phase II and Follow-up 
of Phase | Audit 


Main Points 


7a The follow-up to our 1996 audit chapter indicates limited progress in addressing the deficiencies we 
identified in Phase I of the Canada Infrastructure Works Program. Our 1996 observations dealt with the need for 
clearer definitions of program objectives and project selection criteria, as well as the need for improved 
information, including better measurement and reporting of job creation. The main area of improvement has been 
environmental assessments; as well, a start has been made in responding to the need for compliance audits. 
However, most of our concerns have not been addressed. 


17.2 The Program was implemented within very demanding time frames. Federal and provincial program 
managers view the program approach positively and regard it as having contributed to strong federal-provincial 
relations and co-operation. The Program’s output from 1994 to date has involved mobilizing and co-ordinating the 
efforts and resources of three levels of government and other partners to plan and implement more than 17,000 
engineering and construction projects nation-wide. 


iB Our audit of Phase II indicates that, from an overall federal perspective, the Program is essentially 
“running on trust” with little accountability. Criteria for project selection were not clearly defined, and many of 
the files we reviewed lacked persuasive evidence to justify applicants’ claims relating to selection criteria. In most 
cases, federal officials recommended projects for approval without ensuring that applicants’ claims were 
adequately supported. 


17.4 We found that federal officials relied on municipal and provincial certifications with respect to costs 
claimed. The implementation of compliance audits, which, among other things, represent a means of obtaining 
assurance on the adequacy of financial controls, has been slow. 


17.5 There are incentives for provincial governments to transfer their budgetary resources away from their 
own programs to the federal infrastructure initiative. However, this program substitution is not inevitable; in two 
provinces, for example, safeguards were put in place in Phase I to limit the “substitution effect”. Any substitution 
that does occur reduces the Program’s infrastructure development and job creation benefits. The Treasury Board 
Secretariat has not set out the limitations of estimates of employment generated by the Program in reporting them 
to Parliament. 


Background and other observations 


17.6 The Canada Infrastructure Works Program was introduced by the federal government in 1994 as a 

$6 billion temporary shared-cost initiative with the objectives of assisting in the maintenance and development of 
infrastructure in local communities and the creation of employment. In 1997, the government announced an 
extension (Phase II), involving an additional $425 million provided by the federal government, to be matched by a 
further $850 million from the provinces and other partners. For the most part, construction under both phases was 
scheduled to be completed by 31 March 1999. 


Ee Construction of roads, bridges, and water and sewer networks predominated in both phases, accounting 
for well over 60 percent of total expenditures. Community, cultural and recreational services were of less 
significance in Phase II. 
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17.8 The Canada Infrastructure Works Program is a contributions program, with a requirement for the 
payment of federal funds to be subject to performance conditions being met (such as project targeting) and to 
compliance with program requirements. However, our examination of the project files indicated that program 
officials have not adequately addressed these concerns. 


17.9 Finally, we found that progress has been made under Phase II in conforming with the requirements of the 
Canadian Environmental Assessment Act. However; where negative environmental effects have been identified, 
the specification and monitoring of mitigation measures remain inadequate. 


The Treasury Board Secretariat indicated that although improvements can and should be made in any 
future such programs, the chapter presents an inappropriately negative view of what was a highly successful 
program in terms of results and one that made a positive contribution to federal-provincial-municipal 


relations in Canada. 
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Introduction 


Objectives and structure of the 
Program 


17.10 The Program was introduced in 
1994 to help develop local 
infrastructure and create employment. 
The Canada Infrastructure Works Program 
was implemented as a temporary 
shared-cost program in 1994 with the 
objectives of assisting in the maintenance 
and development of infrastructure in local 
communities and the creation of 
employment. For the purposes of the 
Program, “infrastructure” was defined as 
“physical capital assets in Canada 
instrumental in the provision of public 
services.” 


17.11 The Program is a collaborative 
arrangement. The federal government is 
one of several program partners. The 
others include provincial and local 
governments, and, in some cases, the 
private sector. For the purposes of this 
program, the federal government entered 
into a separate agreement with each 
province and territory. Under these 
agreements, the federal government 
contributes up to one third of eligible 
project costs, and the provincial and local 
governments and other partners contribute 
the remainder. In the majority of cases, 
municipalities identified their priorities 
for infrastructure program funding based 
on local requirements, and submitted 
projects for provincial review. Provinces 
then forwarded projects selected to the 
federal government for review and 
approval. 


17.12 While mostly similar in content, 
the federal-provincial agreements vary 
somewhat according to province or 
territory. Generally, they each set out the 
Program’s purpose, the criteria for project 
selection, and the financial and 
operational responsibilities of the parties 
involved. The provinces are responsible 
for selecting, analyzing and proposing 


Canada Infrastructure Works Program: 
Phase II and Follow-up of Phase I Audit 


projects for funding. The federal 
government has the right to approve or 
reject individual projects proposed. 


17.13 In accordance with the 
agreements, a joint federal-provincial 
management committee was established in 
each province to carry out the 
responsibilities of the federal and 
provincial governments. The management 
committees developed program guidelines 
for the implementation of the Program in 
their respective provinces. 


17.14 Although developing and 
maintaining local infrastructure fall 
essentially under the jurisdiction of 
provinces and municipalities, over the 
years the federal government has 
frequently provided financial aid. This 
assistance has taken the form of loans, 
unconditional transfers of funds (grants) or 
conditional payments (contributions). 


The federal 
government is one of 
several program 
partners investing a 
total of $8.3 billion. 


17.15. The Canada Infrastructure Works 
Program is a contributions program. This 
means that the federal government pays if 
performance conditions and program 
requirements are met, as specified in the 
agreements. These requirements have 
important implications for the Program’s 
accountability regime — in terms of 
accountability among the partners to the 
agreements and also accountability to 
Parliament for expenditures undertaken 
and results achieved. 


17.16 Originally, the federal 
government agreed to contribute up to 

$2 billion, to be matched by $4 billion 
from provincial and municipal 
governments and other partners. In 
January 1997, the government announced 
a program extension (referred to as 

Phase II) whereby it provided an 
additional $425 million to support new 
infrastructure projects, to be matched by a 
further $850 million from the provinces 
and the other partners. Over the course of 
the two phases, all of the partners are 
expected to have invested a combined 
total of $8.3 billion. Under both phases, 
construction for most projects was 
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Exhibit 17.1 


scheduled to be completed on or before 
31 March 1999. 


17.17 Six federal departments and 
agencies take part in the Program. The 
lead minister, identified as the 
Infrastructure Minister, is the President of 
the Treasury Board. Within the Treasury 
Board Secretariat, an Infrastructure Office 
performs a co-ordinating and 
information-generating role. While each 
program within the provinces is fairly 
autonomous, all projects, regardless of the 
value of the federal contribution, require 
the formal approval of the Infrastructure 
Minister. 


17.18 The federal regional development 
agencies — Western Economic 
Diversification Canada, Industry Canada, 
Canada Economic Development for 
Quebec Regions and the Atlantic Canada 
Opportunities Agency — implement the 
programs in their respective provinces. 
Representatives from each of these 


Federal Allocation of Funds Under Phase II 


Recipient 


Federal Allocation 


British Columbia 
Alberta 
Saskatchewan 
Manitoba 

Ontario 

Quebec 

New Brunswick 
Nova Scotia 

Prince Edward Island 
Newfoundland 
Northwest Territories 


Yukon 


Indian and Northern Affairs Canada 


$51,733,000 
34,739,000 
11,584,000 
13,688,000 
153,020,000 
111,645,000 
11,035,000 
14,272,000 
2,373,000 
10,365,000 
1,140,000 
408,000 


6,874,000 


organizations sit as federal co-chairs on 
their respective management committees 
established under the federal-provincial 
agreements. In some provinces, while 
there can be several provincial 
implementing departments, only one 
provincial representative sits as the 
provincial co-chair. Thus, for each 
province there is a balanced 
federal-provincial representation, with one 
co-chair each. In Alberta, a third co-chair 
was appointed to represent municipalities 
in an ex officio role with no formal 
decision-making authority. 


17.19 In addition to the 
above-mentioned agencies, the Canada 
Infrastructure Works Program includes the 
First Nations Infrastructure Initiative, 
which is administered entirely by Indian 
and Northern Affairs Canada and has no 
direct provincial government involvement. 
Hence, there are no federal-provincial 
agreements for this initiative; applications 
for projects are made directly to Indian 
and Northern Affairs Canada. 


The allocation of responsibilities 
between the federal and provincial 
governments is based on their 
comparative expertise 


17.20 The design of the Program 
recognizes that, in general, the provinces 
and municipalities are responsible for 
investments in local infrastructure. At the 
provincial and municipal levels, there is 
extensive experience in planning, 
financing and implementing such 
investments. 


17.21 In both phases of the Program, 
federal funds were allocated to the 
provinces, territories and First Nations 
based on their respective shares of 
population and unemployment (see 
Exhibit 17.1 for Phase II allocations). In 


(First Nations) 


each province and territory, and in Indian 
| Total: $422,876,000* and Northern Affairs Canada, funds were 

allocated in a manner that reflected local 
priorities. This resulted in a variety of 
allocation methods. 


*The difference from $425 million is due to administration costs. 


Source: Infrastructure Office 
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17.22 Based on their specific priorities 
and local requirements, individual 
municipalities and other organizations 
submitted project applications to the 
responsible provincial ministry or, in some 
cases, the joint federal-provincial 
secretariat. If approved at that level, 
applications were forwarded to the federal 
co-chair and the Infrastructure Minister 
for approval. 


17.23. In general, once the Infrastructure 
Minister gave final approval for a project, 
a separate local agreement was made 
between the province or territory and the 
municipality or other organization. These 
local agreements are project-specific, 
dealing with such matters as eligible costs 
and contribution limits; they do not 
involve the federal government directly. 
First Nations projects are characterized by 
local agreements between Indian and 
Northern Affairs Canada and the relevant 
First Nations community. 


Focus of the audit 


17.24 This audit includes a follow-up to 
the observations and recommendations 
presented as lessons learned in the 1996 
Report of the Auditor General on Phase I 
of the Program, as well as an audit of 
Phase II. Detailed testing for this audit 
was confined to the $425 million Phase II 
federal expenditure. Observations and 
conclusions relating to the follow-up 
component are clearly identified. 


17.25 The follow-up component is a 
standard practice of our Office. We chose 
to undertake an additional audit of 

Phase II for several reasons. One 
important reason is the large expenditure 
involved — $425 million. As well, this 
audit takes into account experience gained 
since the first audit. In 1996, the Program 
had been under way for less than two 
years. At the time of our examination for 
this audit, the Program had been in place 
for more than four years. Finally, and 
perhaps most important, this audit is an 
opportunity to identify those practices that 
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have worked well and those that require 
improvement. 


17.26 Objectives. Specifically, this 
audit set out: 


e to determine whether the projects 
undertaken in Phase II conform with the 
targeting and project selection 
requirements of the relevant Infrastructure 
Works agreements; 


e to determine the adequacy of the 
financial and management regime for 
Phase II; 


e to determine the extent to which 
Phase II conforms with the requirements 
of relevant environmental law; and 


e to determine the extent to which the 
observations and recommendations made 
in the 1996 Report of the Auditor General, 
which could have been acted upon in the 
program extension, have been acted upon. 


17.27. Joint undertaking. We 
undertook some of our Phase II audit work 
jointly with the Offices of the Auditor 
General of Nova Scotia and the Provincial 
Auditor of Saskatchewan. As the Program 
is a shared federal-provincial 
responsibility, doing audit work jointly 
provided an excellent opportunity for a 
more comprehensive examination in these 
two provinces. The perspective developed 
also guided a more in-depth coverage of 
the Program in the other provinces where 
we carried out the audit. Along with our 
fellow members of the Canadian Council 
of Legislative Auditors, we believe that 
taxpayers are better served when a total 
audited picture of a program’s 
expenditures is provided. 


17.28 This chapter presents 
observations, conclusions and 
recommendations pertinent to the federal 
government, our specific jurisdiction. The 
reports of the Auditor General of Nova 
Scotia and the Provincial Auditor of 
Saskatchewan are expected to be tabled 
with their respective legislatures, and will 
deal with matters pertinent to the 
provincial governments concerned. 


Some audit work was 
undertaken jointly with 
provincial audit 
offices. 
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17.29 We supplemented the findings 
from our jointly undertaken audit work in 
Nova Scotia and Saskatchewan by 
focussing the rest of our audit on selected 
provinces. Overall, we selected British 
Columbia, Alberta, Saskatchewan, 
Ontario, Quebec and Nova Scotia for audit 
to ensure representation of all federal 
implementing departments. This coverage 
amounted to about 90 percent of the 
Program’s expenditures. For the other 
provinces and territories, the findings 
from our audit do not necessarily apply 
due to possible program differences. 


17.30 For the provinces covered by our 
audit, we examined a sample of federal 
project files and many of the 
corresponding provincial project files. As 
well, in selected cases, we broadened our 
review to include an examination of 
related project files held by municipal 
governments. We examined the First 
Nations Infrastructure Initiative in terms 
of its planning and approval process only. 


17.31 Further details on the audit are 
presented in About the Audit at the end 
of the chapter. 


Observations and 
Recommendations 


Program Implementation 


Some program differences exist between 
Phase I and Phase II 


17.32 In design, structure and 
implementation, Phase II of the Program 
was a continuation of the basic approach 
developed in Phase I. However, Phase II 
did contain some modifications. 
Subsequent to our 1996 Report, 
federal-provincial agreements were 
amended in 1997 to add provisions for 
compliance audit. Stronger controls were 
added to ensure that federal environmental 
assessment requirements were met prior to 
federal approval. Further, in one of the 
provinces we audited, Phase II was 


targeted more clearly on traditional 
infrastructure. 


17.33. Moreover, in Phase II the federal 
government significantly lowered its level 
of expenditures. The federal-provincial 
agreements authorizing additional funding 
for Phase II specified that construction 
activities had to be completed within one 
year. The period for project completion 
was subsequently extended to two years, 
compared with the five years allowed in 
Phase I. The shorter lead times in Phase I], 
coupled with a lengthy approval process 
(see paragraph 17.39), resulted in many 
projects receiving project approval after 
construction had begun. 


Both phases of the Program were 
implemented quickly 


17.34 As we observed in our 1996 
Report, the speed with which Phase I of 
the Program was implemented compared 
favourably with similar collaborative 
arrangements. Phase II was also 
implemented quickly, with most provinces 
signing amending agreements within a 
few months of its announcement. When 
the new one-year time frame for 
completion of construction was seen to 
create difficulties for implementation, the 
Infrastructure Minister announced a 
one-year extension in August 1997. 
However, project contracts between the 
provinces and local municipalities could 
not reflect the new completion dates until 
the federal-provincial agreements were 
amended several months later. 


Program officials view the program 
approach very positively 


17.35 Provincial and federal program 
officials in every province that we audited 
declared the Program a success in creating 
positive federal-provincial relations and 
directing funds to needed infrastructure 
investment. From early 1994 to date, the 
Program has mobilized and co-ordinated 
the efforts and resources of three levels of 
government (as well as the other partners 
involved) to plan and implement more 
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than 17,000 engineering and construction 
projects nationwide. 


17.36 Officials believe that the 
Program’s approach — combining the 
specific expertise of the two levels of 
government within an established 
framework of federal-provincial 
agreements — has been largely 
responsible for this achievement. The 
provincial governments are responsible for 
the day-to-day running of the Program, 
while the federal government has a less 
operational role. Consequently, the federal 
government has not become involved in 
project nomination or in second-guessing 
the decisions made at other levels on the 
screening and selection of initial project 
proposals. 


17.37 Program managers we spoke to 
believed that there were significant 
savings as a result of reduced overlap and 
duplication of activities between the two 
levels of government. Such savings have 
not been measured to date. Moreover, any 
savings identified would need to take into 
account our assessment, as identified in 
this chapter, of the current financial 
management and control arrangements. 


Effort was devoted to ensuring that 
participation of all partners was 
recognized 


17.38 The federal-provincial 
agreements provided for the careful 
co-ordination of joint public 
announcements and the posting of signs 
identifying projects as joint 
federal-provincial ventures. Subsequently, 
considerable administrative effort was 
directed to ensuring that these provisions 
were carried out. This recognition of the 
partners’ efforts has helped foster positive 
relationships. 


17.39 
nomination submitted to the Infrastructure 
Minister for approval was forwarded to 


In some provinces, each project 
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the appropriate provincial and federal 
members of Parliament for their input. As 
well, in all provinces, final federal 
approval could not be given until 
environmental assessment requirements 
had been fully met. In the two provinces 
where we examined these processes in 
detail, we found that all of these 
requirements combined added 
approximately two months to the approval 
process. 


The Program has 
mobilized the efforts 
and resources of three 
levels of government 
(and other partners) to 
implement more than 
17,000 construction 
projects. 


17.40 We were told that in some 
provinces applicants could proceed at 
their own risk prior to federal approval 
because of the tight time frames involved. 
However, if for any reason a project was 
not approved, the federal government’s 
share would not be paid. For many of the 
projects we examined, approval was 
received after construction had already 
begun. In our opinion, the fact that 
construction can begin beforehand reduces 
the significance of the role of the federal 
approval process. 


17.41 The lengthy project approval 
process frustrated program officials. In our 
view, it would have been reasonable to 
streamline the delegation of approvals 
based on risk and level of expenditure, so 
that the Infrastructure Minister need not 
have approved every project application. 
In all likelihood, this would have 
improved the turn-around time for 
approvals. 


17.42 In future programs of this type, 
the project approval process should be 
based on a more streamlined approach, 
with delegation of authority for 
approvals based on risk and level of 
expenditure. 


Treasury Board Secretariat’s response: 
In principle, streamlining of the project 
approval process is desirable. In practice, 
the process is one of the elements that 
must be negotiated with the other program 
partners and we would endeavour to do 
SO. 
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The construction of 
roads, bridges, and 
water and sewer 
networks has 
predominated in both 
phases of the 
Program. 


Most project files we 
examined lacked 
persuasive evidence to 
support the claims of 
project applicants 
relating to selection 
criteria. 
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Funding, Targeting and Impacts of 
the Program 


Funds were spent largely on traditional 
infrastructure 


17.43 The federal government has 
indicated that, with the additional funding 
under Phase II, overall infrastructure 
investments for the Program will exceed 
$8.3 billion — with the federal 
contribution amounting to $2.4 billion. As 
shown in Exhibit 17.2, the construction of 
roads, bridges, and water and sewer 
networks has predominated in both phases 
of the Program. However, there has been 
less emphasis on community, cultural and 
recreational services in Phase II. In both 
phases, there has been a diversity of 
project size and type. In Phase I, for 
example, the largest project is a 

$45.8 million renovation and addition to 
the Cummer Home for the Aged in 
Toronto, Ontario. At the other end of the 
spectrum, $322 was spent on picnic tables 
for the community of Birch Cove, Alberta. 


17.44 Projects undertaken by First 
Nations communities differed somewhat 
from those in the rest of the country, 
owing to distinct community differences. 
There are over 600 First Nations, 

75 percent of which are located in 
communities of less than 1,000 people. 
Most of these communities can be 
described as rural or remote, with vastly 
different infrastructure needs from those 
of other Canadian communities. First 
Nations projects often focussed on 
building or expanding their community’s 
basic infrastructure. 


Program targeting is weak 


17.45 As we did for Phase I, we 
examined the basis on which federal 
officials recommended the approval of 
individual projects. Specifically, we 
looked for evidence that federal officials 
had obtained assurance that their 
provincial counterparts were receiving and 
analyzing supporting information to help 


ensure that the projects approved were in 
line with the Program’s objectives. 


17.46 As we found in 1996, the project 
selection criteria most frequently cited in 
support of project proposals related to 
providing additional infrastructure 
facilities in response to local needs, 
upgrading existing infrastructure to meet 
community standards, and creating 
short-term employment (see Exhibit 17.3). 
In 1996 we noted that the meanings of 
many of the terms relating to these and 
other selection criteria were not clear. We 
found that situation unchanged. 


17.47 We found that the operational 
definition of the term “incremental 
investment” can differ across provinces 
and is not always fully in line with the 
Program’s overall objectives. Similar 
concerns apply to the meaning of 
“employment creation”. We found no 
clear identification of what “community 
standards” mean in practice or how such 
standards could be assessed in applying 
project selection criteria. Finally, as noted 
in paragraph 17.54, even the term 
“infrastructure”, which determines the 
basic scope of the Program, would benefit 
from clarification. 


17.48 Most of the federal and 
provincial project files we examined 
lacked persuasive evidence to support the 
claims of project applicants relating to 
selection criteria. Although some files for 
large and complex projects contained 
more detailed analyses, most applications 
were prepared in qualitative and often 
vague terms, with no information, other 
than certifications by project applicants, to 
back up claims that criteria were being 
met. In most cases, federal officials 
endorsed provincial assessments without 
ensuring that provincial officials had 
received and analyzed the appropriate 
information or requiring direct supporting 
information themselves. 


17.49 In our view, the federal 
government accepted the responsibility for 
approving projects and is accountable to 
Parliament for program results. 
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Exhibit 17.2 


Total Expected Investments by Type of Infrastructure: Phase | and Phase II 


Phase I 
Total Expenditure = $6,806 million 


Phase II 
Total Expenditure = $1,528 million 


Infrastructure Category 


Water and Sewer 


Roads and Bridges 
Municipal Buildings 
Other Buildings* 


| Community Cultural 


Recreation Facilities 


Education Facilities 


Other Engineering and 
Equipment** 

| | | | | | 

100 80 60 40 20 0 0 400 800 1200 1600 2000 2400 


Percentage Distribution Millions of dollars 


* Other Buildings includes various structures such as research and development centres, agricultural facilities, labs, hospitals, 
day-care centres and women’s shelters. 


** Other Engineering and Equipment includes docks, piers, dams, landfills, recycling bins, emergency vehicles, communications 
systems, and other works. 


Examples of Small and Large Phase II Investments Under the Agreements Examined 


Small 


e¢ Replacement of picnic tables in recreational area, Birch Cove, e Widening and rehabilitation of Idylwyld Drive in Saskatoon, 


Large 


Alberta ($322) 

Repair of a well, Duff, Saskatchewan ($500) 

Rehabilitation of a gravel street, Highgate, Ontario ($1,211) 
Band office addition, West Point, Indian and Northern Affairs 
Canada ($39,690) 

Repair of Trunk Road Bike Route, Maple Ridge, British 
Columbia ($2,563) 

Assessment of water and sewer systems, St-Adolphe-d’ Howard, 
Quebec ($10,640) 


Extension of waterline to South River serving a commercial area, 
Antigonish, Nova Scotia ($18,000) 


Saskatchewan ($3.7 million) 

Water and sewer site development, Peguis, Indian and Northern 
Affairs Canada ($4.9 million) 

Design and alignment of a collector road, Halifax, Nova Scotia 
($4.7 million) 

Rehabilitation of residential roads, sidewalks and curbs, 
Edmonton, Alberta ($7.7 million) 

Purchase of rapid transit buses, BC Transit, British Columbia 
($21 million) 

Re-construction of the Dorval traffic circle, Dorval, Quebec 
($36 million) 

Renovations and additions to Toronto’s Cummer Home for the 
Aged, Ontario ($45.8 million) 


Source: Infrastructure Office, January 1999 
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Consequently, there was a need for federal 
officials to make recommendations on 
proposals either on the basis of assurance 
that their provincial counterparts had 
carried out a review of adequate 
supporting information or on the basis of 
their own review. While some measure of 
assurance would be required for every 
project, the depth and detail of review 
could be expected to vary, depending on 
risk and the level of expenditure for a 
particular project. 


17.50 In future programs of this type, 
the government should ensure that 
project selection criteria are clearly 
defined, and that persuasive 
information and analyses are available 
and have been assessed to support 
recommendations for project approval. 


Treasury Board Secretariat’s response: 
We agree that project selection criteria 
should be clear in any further program 
and that projects would need to be 
assessed against them. 


17.51 ‘In the case of Indian and 
Northern Affairs Canada, program 
officials told us that there was a 
requirement that the First Nations 
Infrastructure Initiative should accelerate 
the implementation of supported projects 
by at least three years — the 
incrementality requirement. Once 
approved, First Nations infrastructure 
projects were administered under existing 
departmental capital program activities. 


The implications of 
private-sector-related projects need to 


be addressed 


17.52 As noted earlier, for the purposes 

of the Program, “infrastructure” means 
Exhibit 17.3 “physical capital assets in Canada 
instrumental in the provision of public 
services.” Under Phase I, this broad 
definition was occasionally used to justify 


Frequency of Program Criteria Selected and of Claims Supported 
by Documentation Under Phase II 


eats 5 : investment assistance to firms and 
Applications Claims with d : . : 
Where Supporting corporations involved in private sector and 
Claimed Documentation quasi-private-sector activities. Under 
Criteria (percent) (percent) Phase II, this definition of infrastructure 
Shen miomcenion 98 3 has continued to be applied. This has 
resulted in support for some projects 
Modemization of indrastructure/ involving private sector partnerships and 
brings up to community standards 90 5 : : 
other linkages to the private sector. 
Investment is accelerated/incremental 85 4 e Nhs 
17.53. As part of our audit of Phase II, 
| Improve environmental stability/quality 40 12 we conducted a review of project 
Increase economic competitiveness 34 5 descriptions in the Infrastructure Office 
database, and examined project files 
Long-term job creati es eee F 
ee ae eae ae vi ui where the description indicated possible 
Innovative financing used 16 7 private sector connections. In addition, we 
obtained management representations on 
Long-term worker skills enhanced 10 0 : 
the nature and extent of such investments. 
Innovative technologies used 5 2} We extended our examination to any files 


; ee ; weer, identified by this means. 
The project criteria included above comprise the full range of criteria for the 


Program as a whole. However, it should be noted that some of the provinces did 17.54 We found that federal 
not incorporate all of the criteria in their programs as established by the 


: ae government program expenditures in 
federal-provincial agreements. ss 


support of the projects we identified were 
approximately $10 million. This included 
funding such diverse projects as the 


Source: Office of the Auditor General of Canada; File review (federal and 
provincial project files) 
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construction of a research laboratory and 
tourist facilities (including the 
construction of a convention centre and an 
artificial lake), and upgrading a road 
leading to a mine (see Exhibit 17.4). 
Although these investments made up only 
about two percent of total federal funding 
under Phase II, in our judgment the design 
of the Program would have benefited from 
more precise clarification at the outset of 
the coverage of infrastructure assistance 
that was intended, as well as possible 
implications. 


17.55 For example, our examination 
raised the possible issue of unfair 
competition resulting from direct or 
indirect assistance to some firms and not 
to others. Such government assistance 
may unfairly tilt the playing field against 
other competing firms that do not benefit 
from the assistance provided by the 
Program. In our examination of the 
program guidelines and federal-provincial 
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agreements, and our interviews with 
program officials, we did not find any 
evidence that program designers had 
considered these or other implications of 
funding projects involving private sector 
partnerships and other linkages to the 
private sector. 


17.56 In future programs of this type, 
the government should clearly define 
the coverage of the term 
“infrastructure’’. If that coverage allows 
for support of projects involving private 
sector partnerships and other private 
sector linkages, the government should 
ensure that program guidelines 
specifically address related implications. 


Treasury Board Secretariat’s response: 
The recommendation implies that the term 
“infrastructure” was not clearly defined. 
It was, and it did allow for support of 
private sector initiatives. However, care 
was taken to ensure that none of the 
projects that had private sector 
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Phase II Projects Involving Private Sector Partnerships and Other Private Sector Linkages 


British Columbia: 


e B.C. Telephone Company: installation of basic cable telephone service using fibre optic feeder and copper distribution cables. 
Canada: $121,500, B.C.: $121,500, B.C. Telephone: $550,000 to be partly offset by a surcharge to residents. (Note: There are five 


other B.C. Telephone projects valued at $3.2 million.) 


Ontario: 


e Agrium Mines Limited: 40-km road upgrading to provide access initially for construction of a phosphate mine/mill and for personnel 
access and service vehicles during operation. Canada: $1.7 million, Ontario: $1.7 million, Agrium: $1.7 million. 


e Sunnybrook Health Science Centre: construction and renovation of two floors of an existing facility with a private-sector partner to 
build research laboratory facilities to develop cancer vaccines. Canada: $3 million, Ontario: $3 million, Sunnybrook Health Science 


Centre: $3 million. 


Quebec: 


e Centre Nouvel-air Matawinie Inc.: development of a recreational tourist facility on Lake Taureau including the construction of an 
artificial lake, buildings, pathways and trails, and parking facilities. Canada: $1.3 million, Quebec: $1.3 million, Centre Nouvel-air 


Matawinie Inc.: $1.3 million. 


e Place de la Cité Internationale Phase III Inc.: construction of two underground passages to the Cité Internationale de Montréal. 
Canada: $3.2 million, Quebec: $3.2 million, Cité Internationale: $2.1 million. 


e Société structurante de Shawinigan inc.: construction of a convention centre in the City of Shawinigan to assist tourism sector. 
Canada: $657,678, Quebec: $657,678, City of Shawinigan: $431,290, Société structurante de Shawinigan inc.: $400,000. 


Nova Scotia: 


¢ Meat Cove Fisherman’s Association: wharf repairs. Canada: $8,333, Nova Scotia: $8,333 and Meat Cove Fisherman’s Association: 


0,335: 
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involvement (i.e. two percent of federal 


funding under Phase IT) resulted in unfair 


competition. 


Incentives exist to substitute Canada 
Infrastructure Works Program funds 


programs, ongoing financial support to 
municipalities for local infrastructure 
investment. During the 10-year period 
before the introduction of the Program, 
ongoing construction and repair 
expenditures at the local municipal level 


for regular, ongoing provincial 
infrastructure expenditures 


averaged $5.6 billion annually. 


17.58 Exhibit 17.5 outlines examples of 
provincial government programs 
providing ongoing assistance to 
municipalities for infrastructure 
investment, based on programs in place 


17.57 Besides the assistance provided 
under the Canada Infrastructure Works 
Program, provincial governments have 
been providing, through their own 


Exhibit 17.5 


Local Infrastructure Investment Assistance 
Provincial Government Cost-Share Ratio (%) Under Canada Infrastructure Works Program 
and Other Selected Provincial Capital Assistance to Municipalities 


1993-1999 
Provincial Government Cost-Share Ratio 
Canada Infrastructure 
Works Program * Selected Provincial Government Infrastructure Programs 
(percent) (percent) 
Government of Alberta Basic Capital Grants 75 
Canada—Alberta 33 Street Improvement Program qd 
Infrastructure Works : : 
Secondary Highway Partnership Program 65-85 
Municipal Water/Waste Water Up to 75 
Government of Ontario Municipal Affairs Unconditional Grant 100 
Homes for Aged 50 
Canada Ontane ae Elderly Persons Centres 30 
Infrastructure Works = Educational Capital Grants 60-75 
Grants for Hospital Construction/Renovation 66 
Northern Ontario Industrial Infrastructure 1S 
Roads/Bridges 50-80 
Government of Quebec Canada—Quebec 33 Culverts/Sewage 50 
dn fase cuaie: Nous Water Purification 50-85 
Government of @anadaaNovaeScota 33 Provincial Capital Assistance Program 50 
Nova Scotia Infrastructure Works Unconditional Capital Grants 100 
— ae 


* Projected contribution — actual contribution by provincial government may be somewhat higher or lower. 


Source: Alberta: Ministry of Transportation and Utilities 
Ontario: Provincial Financial Assistance to Municipality Boards and Commissions 1992-93 
Quebec: Ministére des Affaires municipales 
Nova Scotia: Nova Scotia Department of Housing and Municipal Affairs 
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for either some or all of the period from 
1993 to 1999. This covers the period 
immediately before and during the 
implementation of the Canada 
Infrastructure Works Program. In most 
cases, the provinces’ own programs 
require provincial governments to outlay 
half or more of the eligible costs of 
municipal infrastructure investments. 
However, when projects are funded under 
the Canada Infrastructure Works Program, 
provincial government expenditures 
average only about one third of the total 
investment. Because of the different 
provisions of federal and provincial 
programs, we believe there are financial 
incentives for provincial governments to 
transfer their budgetary resources away 
from their own regular programs to the 
national infrastructure program. 


17.59 The Canada Infrastructure Works 
Program is meant to encourage additional 
investments in local infrastructure — a 
condition necessary to increase 
infrastructure development and create 
employment. For this reason, if provinces 
simply switch funds between programs, 
the national program’s intended effects are 
reduced. 


17.60 The government’s own 1996 
evaluation of the Program expressed 
concerns about the lack of effort to 
enforce the requirement to encourage 
additional investment levels. The 
evaluation identified this as a “program 
weakness’. We believe that the possibility 
of program substitution deserves attention 
as a key part of this issue. 


17.61 Identifying the actual extent of 
program substitution that may have 
occurred is an extremely difficult task, 
and one that the current audit could not 
address in the absence of detailed data 
from other program partners. However, in 
our examination of project files, we found 
evidence that some projects approved 
under the Canada Infrastructure Works 
Program may have been funded under 
programs that would have been more 


costly for the provinces involved. 
Although we would not have expected 
project files automatically to contain such 
information, we found specific cases 
where projects that would likely otherwise 
have received approval for funding under 
existing provincial programs, or had 
actually received approval, were 
ultimately funded under the Canada 
Infrastructure Works Program. We believe 
that this file evidence, coupled with the 
financial incentives identified in 

Exhibit 17.5, suggests that program 
substitution is an issue that needs to be 
addressed in program design. 


17.62 The period during which the 
program functioned featured a lot of 
general budget cuts by provincial 
governments, including cuts in provincial 
capital assistance programs. At the same 
time, however, the federal initiative was 
attempting to increase levels of 
infrastructure investment. In this context, 
substitution would represent a rational 
response by provinces to the national 
infrastructure program. It does not 
represent a violation of the Program’s 
terms and conditions; rather, it responds to 
a design element that program developers 
may not have sufficiently considered 
beforehand. To the extent that it occurs, 
program substitution directly affects the 
achievement of the Canada Infrastructure 
Works Program’s primary objectives: 
infrastructure development and 
employment creation that would not have 
occurred otherwise at that time. 


Safeguards are needed to limit program 
substitution 


17.63 We examined the provisions 
established within federal and provincial 
programs to identify those that could act 
as safeguards to limit substitution between 
federal and provincial programs. We 
found such provisions to be the exception 
rather than the rule. 


17.64 The types of provisions used in 
Quebec and by the Saskatchewan 
Association of Rural Municipalities in 


There are financial 
incentives for 
provincial 
governments to 
transfer budgetary 
resources from their 
own regular programs 
to the national 
infrastructure 
program. 


Program substitution 
directly affects the 
achievement of the 
Program’s primary 
objectives: 
infrastructure 
development and 
employment creation. 
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How much job creation 
can be attributed to 
the Program’s 
activities remains 
unclear. 


Phase I of the Program provide examples 
of safeguards that can deter program 
substitution. In 1996, we reported that 
Program expenditures in Quebec had to be 
in addition to a minimum level of ongoing 
investment that municipalities were 
required to maintain in order to be eligible 
for assistance under the Program. In these 
circumstances, we believe that 
substitution would be necessarily limited. 


17.65 In future programs of this type, 
the government should ensure that there 
are safeguards to limit the substitution 
of program expenditures for 
expenditures that would otherwise have 
been undertaken by the programs’ 
partners. 


Treasury Board Secretariat’s response: 
While not disagreeing with the 
recommendation per se, the government 
disagrees with the implication that there 
were no safeguards to limit substitution in 
Phases I and II. There were and they 
worked: after two audits and one 


follow-up, no evidence of significant 


substitution has been presented. 


Continuing questions about job creation 
estimates 


17.66 The federal government claims 
that about 130,000 short-term jobs have 
been created since the Program began in 
1994; 24,000 of these are associated with 
Phase II. The estimate of the short-term 
employment impacts 1s basically derived 
from Statistics Canada’s data on the 
employment effects associated with given 
levels of investments. The Statistics 
Canada figures are based on the historical 
track record of investment expenditures in 
generating direct employment, both 
on-site (where the infrastructure is being 
developed or upgraded) and off-site 
(supplies and services provided). 


17.67 Our 1996 audit recognized a 
number of strengths derived from the 
application of the Statistics Canada 
employment estimation model. However, 
we also questioned whether it was 
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appropriate to rely on this approach alone. 
We noted that the assessment of the 
Program’s employment impact would be 
more balanced if more than one 
estimation approach were used, 
particularly for individual projects at the 
local level. 


17.68 We found that the government 
has not taken any steps either to improve 
the estimates of employment actually 
created by the Program, or to clearly set 
out the limitations of employment 
estimates in reporting them to Parliament. 
The Treasury Board Secretariat’s 
Performance Report to Parliament for the 
period ending 31 March 1998 indicates 
simply that over 128,000 jobs have been 
generated by the Program. It provides no 
information about such matters as the 
source of the estimates, the duration of the 
jobs or whether all these jobs can be 
attributed to the Program. 


17.69 The answer to the question of 
how much job creation can be attributed 
to the Program’s activities remains 
unclear. If, for example, the Program’s 
funding of infrastructure investment 
resulted in part in substituting Canada 
Infrastructure Works Program 
expenditures for regular, ongoing 
provincial program expenditures, then the 
Program’s net job creation effects would 
be correspondingly reduced. 


17.70 At the time of our examination, 
we noted a project where Canada 
Infrastructure Works Program funds were 
supplemented by another federal program. 
Yet until we brought this to the attention 
of program management, all of the 
estimates of employment benefits were 
attributed to the Canada Infrastructure 
Works Program alone. Subsequent 
changes were made to the project that 
dealt with the issue we identified. Where 
other programs contribute to an 
infrastructure project, the employment 
associated with that project cannot be 
attributed solely to the Canada 
Infrastructure Works Program. 


Report of the Auditor General of Canada — September 1999 


17.71 In our judgment, the absence of a 
more thorough assessment of the 
Program’s job creation effects is a crucial 
omission. The lack of such an assessment 
erodes the reliability of program 
performance information, which is 
essential for adequate accountability to 
Parliament and Canadians. Furthermore, 
without adequate performance information 
there is a risk that program design features 
of questionable effectiveness will be 
repeated, should any decision be made to 
undertake a similar initiative in the future. 


17.72 Inits Performance Report, the 
Treasury Board Secretariat should 
provide Parliament with information on 
the employment effects of the Canada 
Infrastructure Works Program that 
clearly sets out its sources and 
limitations. 


Treasury Board Secretariat’s response: 
We agree that Parliament should be 
provided with complete information on the 
employment effects of the Canada 
Infrastructure Works Program. 


Financial and Management 
Controls 


Insufficient attention to ensuring 
control 


17.73 As noted earlier, the Canada 
Infrastructure Works Program is a 
contributions program. The payment of 
federal funds is therefore dependent on 
performance conditions being met and on 
compliance with program requirements, 
which include conformity with program 
targeting and appropriate control of public 
funds. 


17.74 Our current audit has confirmed 
and extended the findings from our 1996 
Report: from an overall federal 
perspective, the Program is essentially 
“running on trust”, with insufficient 
attention given to ensuring adequate 
control of public funds. In several of the 
provinces we examined, public moneys 
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have been released to municipalities for 
work undertaken without provincial 
officials demanding invoices or proof of 
payment (see Exhibit 17.6). Without this 
documentation, it is impossible to 
determine whether claims have been made 
for expenditures specifically disallowed 
by the Program, or even whether the 
expenditures were actually made. 


17.75 In every province examined, we 
observed that federal officials relied on 
provincial and municipal certifications 
that the projects were implemented and 
that costs were claimed in accordance 
with the federal-provincial agreements 
and guidelines. The implementation of 
compliance audits, which represent a 
means of verifying certifications, has been 
slow. Compliance audits have almost 
always taken place after all projects were 
approved, and in many cases after projects 
were completely constructed and funds 
paid out. 


From an overall 
federal perspective, 
the Program is 
essentially “running 
on trust”, with 
insufficient attention 
given to ensuring 
adequate control of 
public funds. 


Implementation of compliance audits 
has been slow 


17.76 In our view, timely compliance 
audits are required to provide assurance to 
federal and provincial/territorial partners 
that appropriate financial and 
management controls are in place and 
working as intended. Although we 
recognize that resources are inevitably 
limited, we believe that timely audit is 
particularly important where the up-front 
activities of a program are based on 
certification only. Federal Treasury Board 
policy, with respect to terms and 
conditions for contributions, states: 

“5.2 (b) Where less control is required, the 
description of allowable expenditures may 
be more general and audited assurance 
such as an audit report may be substituted 
for receipts and detailed statements of 
expenditures.” 


17.77 With the exception of Quebec, 
and to a lesser extent British Columbia, 
the Phase II compliance audits are 
scheduled for completion only after a 
considerable amount of time has passed 
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Compliance audits that 
have been undertaken 
illustrate the value of a 
timely and rigorous 
audit regime. 


since the start of the Program. Their 
completion averages approximately two 
and one-half years into the Program — six 
months before most final payments are 
scheduled to have been processed. In our 
view, this delay creates two problems. 
First, if the audits were to identify 
required changes in program approach, the 
opportunity to implement these changes 
might be missed altogether. Second, we 
are In agreement with some program 
officials who maintain that if the audits 
were to identify ineligible costs paid out, 
it would be more difficult to recover these 
moneys, as the final payments (including 
the release of the holdback) would likely 
have occurred for many of the projects. 


17.78 Compliance audit work that had 
been undertaken for both phases of the 
Program at the time of our audit illustrates 
the value of a timely and rigorous audit 
regime. For example, a provincial 
compliance audit of a Phase II project 
identified $121,000 of $516,000 in 
expenses claimed as potentially ineligible. 
Upon resolution by the applicable 
management committee, including 
subsequent changes to the original project 
approval date, $49,000 was determined to 
be ineligible. In still another case, auditors 
raised questions about $279,000 of 

$1 million in expenses claimed. As a 
result of this, the auditors recommended 
that the provincial ministry concerned 


Exhibit 17.6 Province Payment of Claims Compliance Audit 
of Phase II 

Compliance Procedures Used Assurance Verification of Invoices 

Under Phase II (Federal) (Provincial) 

British Columbia No No Ongoing; completed 
for selected projects 
as time and resources 
permit 

Alberta No Only small Planned completion: 

communities; large Fall 1999 
communities that 

represent 65% of 

expenditures are not 

verified 

Saskatchewan No No Planned completion: 
Fall 1999 

Ontario Detailed claim No for most projects; | Planned completion: 

form for most however, varies by Fall 1999 
projects provides | implementing 
information on department 
cost eligibility 
Quebec No Yes for all projects; Ongoing; completed 
ineligible expenses prior to final payment 
deducted from claims. | for selected projects 
In addition, 
compliance audit of 
a sample based on risk 
and materiality 
Maureen OMicHoriie Audie Goeea Nova Scotia No Yes for all projects; Planned completion: 
See a SSee AO ineligible expenses Summer 1999 
of Canada; File reviews, interviews ; ; 
me d ; deducted from claims 
and review of departmental audit plans 
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request supporting documentation for the 
next claim. 


17.79 During our own examination of 
Phase II project files, we noted that both 
levels of government had paid for some 
regular, ongoing provincial and municipal 
worker salaries. Under the Program, for 
these salaries to be eligible, it had to be 
demonstrated that these government 
workers would have been laid off or that 
they were hired specifically as a result of 
the project approved. However, in 25 of 
167 projects examined, approximately 
$2.5 million in salaries was paid out 
without adequate assurance that they were 
incremental. 


17.80 In our view, the federal partner 
needs assurance that all costs claimed and 
paid out are eligible as specified in the 
federal-provincial agreements and 
program guidelines established for each of 
the provinces. Because it may be more 
difficult to recover funds once they have 
been paid out in full, we urge that a timely 
review take place in much the same 
manner as undertaken by the Province of 
Quebec. In this province, a sample of 
projects is selected based on risk and 
materiality, and is audited prior to the 
final payment. If an adjustment is 
required, the amount is deducted from the 
final payment. 


17.81 In future programs of this type, 
the government should ensure that 
agreements among partners make 
adequate provision for rigorous and 
timely compliance audits. Such 
provisions should clearly assign 
responsibilities among partners, and 
specify the coverage, timing and 
reporting of audits, along with resource 
levels to be allocated to the compliance 
audit function. 


Treasury Board Secretariat’s response: 
Compliance audits are a requirement in 
Phase Il and are either completed or 
under way in all provinces. As Canada 
Infrastructure Works Program payments 
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are made from one order of government to 
another, and as these governments have a 
range of ongoing financial transactions, 
the risk of not being able to recover any 
overpayments is minimal. 


Insufficient assurance of value for 
money 


17.82 We found little evidence in the 
project files we reviewed to indicate that 
federal officials had looked for assurance 
that assessments of the technical and 
financial feasibility of project proposals 
had been completed by provincial 
officials. Such assessments would help to 
establish the technical feasibility of the 
infrastructure project proposed and to 
determine whether there are lower-cost 
alternatives and whether the cost estimates 
provided in the application are reasonable. 
In short, such assessments would help 


ensure value for money. In all the provinces 


examined, 
environmental 
assessments were 
undertaken before 
final project approval. 


17.83 The federal-provincial 
agreements have provisions that limit the 
possibility of cost overruns being paid. 
However, we observed several cases 
where amendments had been made to 
cover such extra costs. These cases may 
point to a lack of adequate costing in 
project proposals. In still another instance, 
an internal audit identified an approved 
project with construction standards that 
were beyond provincial standards. In this 
case, the issue was dropped, as matters 
relating to construction standards were not 
identified during the approval stage. With 
an up-front assessment, these costs might 
well have been avoided. 


17.84 When we raised these issues with 
program officials, they claimed that 
establishing such controls for all projects 
would add significantly to program 
operating costs. A possible solution would 
be to use a selective assessment process 
tied to the value of the projects concerned, 
along with a risk assessment based on the 
individual municipality’s track record for 
infrastructure investment. We do not 
suggest that the federal government needs 
to carry out the assessments itself; rather it 
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Selective verification 
of the implementation 
and effectiveness of 
mitigation measures 
would provide 
additional assurance. 


needs assurance that a system is in place 
and operating adequately. 


17.85 The federal-provincial 
agreements specify that contracts must be 
awarded and administered according to 
provincial administrative, management 
and contract procedures. This may or may 
not include the requirement that all 
contracts be tendered. Among the 
provinces we examined, only 
Saskatchewan had guidelines requiring 
project contracts to be tendered. 


17.86 Inthe majority of the federal and 
provincial files we examined, we found no 
information to indicate whether tendering 
had taken place. In a limited number of 
cases, however, partial or indirect 
references in file material showed that the 
projects had been tendered. Overall, 
except in the case of one Ontario 
government implementing department, the 
project files provided little information 
about whether or not this control was 
implemented. 


17.87 In future programs of this type, 
the government should ensure that 
project proposals are assessed to 
provide an adequate level of assurance 
with respect to technical feasibility and 
financial requirements. 


Treasury Board Secretariat’s response: 
The program design takes advantage of 
the fact that local and provincial 
governments have much experience in 
carrying out infrastructure projects and 
that they are paying two thirds of the 
costs. Thus they are both competent and 
motivated to obtain value for money. 
Further detailed monitoring by the federal 
partner would be unlikely to improve the 
return on the taxpayers’ investment. 


Improvements made in fulfilling federal 
environmental obligations 


17.88 Unless excluded under the 
Exclusion List regulations of the 
Canadian Environmental Assessment Act, 
all projects recommended for approval are 


subject to a federal environmental 
assessment in accordance with that Act. 
For the purposes of the Canada 
Infrastructure Works Program, an 
environmental assessment involves 
examining the proposed project to ensure 
that its potential impacts on the 
environment are identified. In those cases 
where potential negative impacts are 
identified, the process requires project 
proponents to specify the measures they 
will take to mitigate them. The federal 
authorities are responsible for ensuring 
that these assessments are undertaken, 
including the identification and 
monitoring of mitigation measures. 


17.89 More than half of all Phase II 
projects did not require environmental 
assessments. Where they were required, 
the federal project files in all of the 
provinces we examined contained 
evidence that they were undertaken before 
final federal project approval. As we 
found in 1996, procedures and 
documentation varied among the 
departments. However, for the most part, 
we noted considerable improvements in 
the thoroughness of documentation and 
analysis undertaken for Phase II projects. 
Unfortunately, we also observed 
weaknesses in the identification and 
monitoring of mitigation measures. 


17.90 Mitigation measures are designed 
to deal with potentially harmful 
environmental effects. Incorporating these 
measures during the construction of 
infrastructure projects increases the 
likelihood that risks to the environment 
are effectively mitigated. Their 
implementation is a necessary condition 
for the receipt of federal funds, and 
requires monitoring. Mitigation measures 
need to be clearly specified, along with 
the project applicant’s responsibility for 
implementing them. 


17.91 The results of our examination 
vary by province, with some project files 
containing little or no information on 
mitigation efforts required and others 
providing very complete information. 
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Project proponents were asked to certify, 
either directly or indirectly, that mitigation 
measures had been implemented. 
However, we found no evidence in the 
project files we examined to show that 
federal authorities had verified whether 
these measures were put in place and were 
effective. We believe that such 
verification, carried out selectively, would 
provide additional assurance that 
irreparable damage has not been done to 
the environment. In this context, we note 
that in most of the provinces we reviewed, 
compliance audits do not address matters 
relating to the implementation of 
mitigation measures. 


17.92. We observed several projects 
where construction began before federal 
approval was given, and before an 
environmental assessment was completed. 
Because of the short time frame in which 
Phase II of the program had to be 
implemented, some provinces informed 
applicants that they could go ahead with 
construction. However, the applicants 
would be responsible for all costs, 
including any environmental clean-up 
costs, if the project was not approved. In 
half of the Ontario projects we examined 
that required mitigation measures, 
construction began before the 
environmental assessment was completed. 
Two of these projects were entirely 
constructed before the environmental 
assessments had been completed. As the 
mitigation measures had not been 
specified when construction began, their 


planning and approval process to be an 
effective means of preventing 
environmental damage. As we observed in 
1996, it may be difficult to withdraw 
support for projects once funding 
commitments — albeit conditional — 
have been made. 


17.94 In future programs of this type, 
the government should ensure that: 


e environmental assessments are 
completed early enough to be taken into 
account in the project planning and 
approval process; 


e necessary mitigation measures are 
clearly identified; and 


e asystem for obtaining assurance of 
the implementation of mitigation 
measures is in place. 


Treasury Board Secretariat’s response: 
We agree that going beyond the specific 
requirements of the Canadian 
Environmental Assessment Act is 
desirable, and that the design of any 
future program should facilitate that. 
However, the project approval process 
would need to retain sufficient flexibility 
to accommodate special situations. 


Response to 1996 Audit 


Limited progress on the 
recommendations of the 1996 Phase I 
audit 


17.95 Weconcluded that the 


The government has 
made limited progress 
in responding to the 
recommendations of 


pier ard 4G ati government has made limited progress in . 
identification and implementation were ) Ps 4 : our 1996 audit of 
left solely to the discretion of the project responding to the recommendations of our 

proponents. 1996 audit of Phase I (see Exhibit 17.7). Phase I. 


17.93 Beginning construction before 
federal environmental assessment and 
project approval does not contravene the 
provisions of the Canadian Environmental 
Assessment Act. However, it contravenes 
the spirit of the Act, as irreparable 
environmental degradation effects are 
possible. We are concerned that, in many 
cases, environmental assessments have not 
occurred early enough in the project 


While some changes were introduced as 
part of the Program’s operational 
guidelines, weaknesses remain with 
respect to the definition of key program 
terms and specification of responsibilities. 
Although federal-provincial agreements 
were amended to establish provisions for 
compliance auditing, the results have been 
disappointing. For the most part, audit 
activities still have not produced timely 
information on compliance with the 
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Program ’s terms and conditions. There has 
been no progress on our 1996 
recommendation relating to the need to 
improve the measurement and reporting of 
job creation effects. Improvements have 
been made in fulfilling federal 
environmental obligations, but more work 
remains to be done. 


Improvements are needed in 
federal-provincial agreements 


17.96 The federal-provincial 
agreements play a pivotal role in the 
implementation of the Program. We 
assessed the adequacy of the current 
agreements in the context of the problem 
areas identified through our audit, and 
also based on a comparison with the 
provisions of other types of 
federal-provincial agreements. 


17.97 We concluded that in any future 
program activities of this type, the 
agreements could be improved in several 
areas. First, the term “infrastructure” 
could be defined more clearly. Also, 
project selection criteria need to be 
clarified, particularly in support of 
objectives relating to generating 
additional infrastructure investment and 
creating employment. 


17.98 Further, continuing concerns with 
the timeliness and coverage of compliance 
audits indicate that a more explicit 
delineation of the responsibilities of the 
federal and provincial partners is needed 
in these areas. 


17.99 Finally, we recognize that the 
federal government took important steps 
during Phase I to implement a program 
evaluation. However, despite evidence of 
deficiencies from this source, as well as 
from our 1996 audit and compliance 
audits, program management has not 
made the necessary adjustments. Hence, 
we believe there is a need to incorporate 
more rigorous and formal provisions 
within agreements to address problems 
revealed by feedback on performance. 
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Conclusion 


17.100 This audit and our 1996 audit 
confirm that both phases of the Canada 
Infrastructure Works Program were 
implemented quickly. There is no doubt 
that participants and managers alike view 
the Program very positively. 


17.101 In this audit, we were particularly 
concerned to determine whether the 

Phase II projects conformed to the 
targeting and project selection 
requirements of the relevant 
federal-provincial agreements. Our 
conclusion 1s that, overall, there is a lack 
of clarity in a number of key targeting 
requirements established as part of the 
Program’s design, including what actually 
constitutes infrastructure or an 
incremental investment for the purposes of 
the Program. 


17.102 The Program’s approach may 
well have produced several benefits; 
however, there is still a need for adequate 
financial control and timely, reliable 
performance information. We have 
concluded that financial and management 
controls for this program are inadequate. 


17.103 Grants are unconditional 
payments made by the federal government 
to individuals and organizations, whereas 
the payment of a contribution is subject to 
performance conditions that are specified 
in a contribution agreement. The recipient 
must show that these conditions are being 
met in order to be reimbursed for specific 
costs over the life of the agreement. The 
government can also audit the use of 
contributions, whereas this is usually not a 
requirement for a grant. While there can 
be some blending of the requirements 
involved, the accountability requirements 
for contributions are generally more 
onerous than for grants. 


17.104 The government specifically 
chose the option of contributions over that 
of grants as a means of delivering the 
Canada Infrastructure Works Program. 
However, we have found that, from an 
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Certain controls need 
to be put in place to 
provide assurance that 
public funds are spent 
economically, 
efficiently and 
effectively. 


overall federal perspective, the Program is 
essentially “running on trust’, with little 
confirmation that performance conditions 
are being met. 


17.105 Although the positive features of 
the present program approach cannot be 
ignored, we believe that certain controls 
need to be put in place to provide 
assurance that public funds are spent 
economically, efficiently and effectively, 
and that accountability for results is 
achieved. We do not suggest that the 
federal government itself needs to 
exercise up-front controls. It does, 
however, need assurance that controls are 
in place and working adequately. In this 
regard, a rigorous compliance audit 
function needs to be implemented in a 
timely manner. 


17.106 The absence of adequate 
financial and management controls for 
this program is not the first case where we 
have identified such problems. Our 
December 1998 Report, Chapter 25 
Transport Canada — Investments in 
Highways, identified similar concerns. We 
believe that if the government puts in 
place a set of conditions, it is reasonable 
to expect it to institute mechanisms to 
ensure adherence to those conditions. 


17.107 We found that progress has been 
made in the extent to which Phase II of 
the Program conforms to the requirements 
of the Canadian Environmental 
Assessment Act. We have concluded that 
the requirements of the Act are being met 
with respect to assessments of 
environmental impacts. However, where 
negative environmental effects have been 
identified, requirements for specifying and 
monitoring the implementation of 
mitigation measures are inadequate. 


17.108 The government has made 
limited progress in responding to the 
recommendations of our 1996 audit of 
Phase I of the Program. Most of our 

Phase I recommendations addressed issues 
that could have been pursued under 

Phase II, but were not. In view of the size 
of the expenditures involved in Phase II 
operations, this lack of progress is a 
source of concern. 


Treasury Board Secretariat’s response: 
Commencing in 1994, Canada 
Infrastructure Works was at the leading 
edge of intergovernmental collaboration 
in the effective delivery of a trilateral 
program. It has resulted in $8.3 billion 
being invested in over 17,000 projects 
across Canada, and involved hundreds of 
governmental organizations each adding 
their particular strengths to keep 
administration costs to a minimum. All of 
this was accomplished without any 
financial mismanagement. 


Unfortunately, the auditors have 
approached their review as if Canada 
Infrastructure Works was a standard 
federal contribution program requiring the 
traditional amount and type of supervision 
above and beyond meeting the legal 
requirements. This fails to recognize that 
each of the three orders of government 
involved had investments at stake, each 
was competent to effect its part of the 
collaborative program delivery and that 
each had audit and evaluation capabilities 
as well as political oversight. 


Although improvements can and should be 
made in any future such programs, the 
chapter presents an inappropriately 
negative view of what was a highly 
successful program in terms of results and 
one that made a positive contribution to 
federal-provincial-municipal relations in 
Canada. 
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y About the Audit 


Objectives 
Our objectives were: 


e to determine whether the projects undertaken in Phase II conform with the targeting and project selection 
requirements of the relevant Infrastructure Works agreements; 


e to determine the adequacy of the financial and management regime pertaining to Phase II; 


e to determine the extent to which Phase II conforms with the requirements of relevant environmental law; 
and 


e to determine the extent to which the observations and recommendations made in the 1996 Report of the 
Auditor General, which could have been acted upon in the program extension, have been acted upon. 


Scope and Approach 


We selected British Columbia, Alberta, Saskatchewan, Ontario, Quebec and Nova Scotia for audit in order to 
ensure geographic representation and coverage of 90 percent of the program’s federal expenditures. We 
examined a sample of federal files and the corresponding provincial files. As well, in selected cases, we 
broadened our review to include an examination of the corresponding municipal project files. All projects that 
received a federal contribution of greater than $1 million (56) were examined, with the remainder selected at 
random and stratified by project class to ensure a comprehensive sample. Altogether, we reviewed 

167 federal, 107 provincial and 12 municipal project files. 


In addition, because of its unique features, we examined the First Nations Infrastructure Initiative strictly in 
terms of its planning and approval processes. We did not examine project files. 


Criteria 
Our audit was based on the following criteria: 


e Actions should have been taken on recommendations made in the 1996 Report of the Auditor General 
that apply to Phase II of the Program. 


e Financial and management control of program expenditures should comply with relevant federal policies. 


e Program management should ensure that a program accountability framework is developed and clearly 
communicated, including assignment of responsibility for the achievement of results. 


e Program management should set out clear operational objectives that define “success” and provide the 
basis for program implementation. 


e Program management should measure, monitor and clearly report on results achieved to legislators and 
within and among federal/provincial partners. 


e Program management should be supported by reliable, relevant and timely information on program 
needs, operations and performance. 


e Program processes and procedures and their implementation should further the achievement of program 
objectives and comply with the terms of the agreements, while controlling costs. 
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e Program management should ensure that each project has been assessed for potential environmental 
concerns. 


e The Program should be implemented in conformity with the requirements of The Canadian 
Environmental Assessment Act, where applicable. 


Audit Team 


Assistant Auditor General: Maria Barrados 
Principal: Henno Moenting 
Director: Jim Blain 


Louise Grand’ maison 
Jayne Hinchliff-Milne 
Raymond Kuate-Konga 
Pierre Labelle 

Joanne Moores 


For information, please contact Henno Moenting. 
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Chapter 18 


Public Works and Government 
Services Canada 


Alternative Forms of Delivery: 
Contracting for Property 
Management Services 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 
Accountants. The numbered paragraphs in bold face represent recommendations. 
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Public Works and Government 
Services Canada 


Alternative Forms of Delivery: 
Contracting for Property 
Management Services 


Main Points 


18.1 The contracting process used by Public Works and Government Services Canada (PWGSC) to contract 
out the operation and maintenance of two thirds of its buildings was well managed and consistent with the 
objectives of the Department. We found that the bidding process was open, transparent and fair. The contracts 
were Well designed and the handover to the contractor went smoothly. A significant feature of the contracting 
process was the use of “fairness monitors” to provide independent assurance to the Department’s senior 
management that the process was conducted fairly and in accordance with the rules. 


18.2 Because some of the key monitoring and control features were not in place at the time of the handover of 
operations, ongoing management of the contracts has been problematic. Important items that had to be ready at 
contract signing — such as budgets for the first year, assessments of building condition and performance 
indicators of tenant satisfaction — were not completed until well into the first year. As a result, the Department 
will not be able to carry out a complete assessment of the contractor’s performance in the first year and thereby 
get full assurance that it is receiving the quality and quantity of service to which it is entitled. 


Background and other observations 


18.3 Public Works and Government Services Canada, through its Real Property Services Branch (RPS), is the 
primary landlord of a significant proportion of the government’s real property holdings. Its inventory amounts to 
about 6 million square metres of space — from office buildings, laboratories and warehouses to the Parliament 
buildings and other “national treasures”. It also administers 3,000 leases representing annual rent of $550 million. 
This makes the Branch one of the largest property managers in the country. 


18.4 In 1996, the Department embarked on an initiative called “Alternative Forms of Delivery” (AFD). A key 
element of the initiative was to contract out the management and operations of a portfolio of buildings, including 
services ranging from cleaning and the operation of building systems to maintenance, landscaping and repair 
projects. This initiative was a response to the government’s 1994 Program Review and served as a strategy for 
restructuring the Branch around its core business activities. The Branch also wanted to support the Canadian 
property management industry and reduce costs through this initiative. 


18.5 The Department contracted out to the private sector the operations and maintenance of approximately 
300 buildings with about 2.3 million square metres of office space. It expects to achieve about $20 million 
annually in cost savings by turning over the management of those buildings to the private sector. 


18.6 The buildings were regrouped by region and offered to the private sector in 13 separate contracts, with a 
total value of around $170 million. The same contractor won each of the 13 contracts. It offered jobs to almost all 
affected employees at full salary for a minimum of three years. About 465 employees accepted the transfer to the 
contractor. 
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18.7 It is too early to assess whether savings the Department expects from this initiative will be achieved. We 
cannot measure at this time whether the Department will receive services at least equal to those it provided itself, 
and receive them at lower cost. However, mechanisms were incorporated in the contracts that are conducive to 
achieving savings. We will follow up on these issues in future audit work. 


18.8 We encourage Public Works and Government Services Canada to continue to use “fairness monitors” for 
large and complex contracts. In a project of this scope and complexity, we did not expect to find that all 
contractual requirements would be fulfilled by the handover date. We did expect that the Department would have 
established an action plan to deal with the uncompleted deliverables. We have recommended that it develop an 
action plan with the contractor to deal with outstanding contractual issues, including target completion dates and 
respective responsibilities for resolving them. 


The Department has agreed with our recommendation. 
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Introduction 


18.9 One of the objectives of the 
government’s 1994 Program Review was 
to “get government right”. An important 
element of this was identifying 
government activities that might be 
handled better by the private sector. For 
its part, the Real Property Services Branch 
of Public Works and Government Services 
Canada (PWGSC) began a process of 
self-examination that ultimately led to its 
“Alternative Forms of Delivery” (AFD) 
initiative, part of which was contracting 
out the management of many of its office 
buildings. 


18.10  Inits 1995 Business Plan, the 
Branch identified its principal role as that 
of “real property advisor” to the 
government. Its long-term strategy would 
be to focus on developing its capabilities 
to fulfil that role, and “to work more 
closely with the private sector in the 
delivery of certain operational and 
transactional services.” 


18.11 One of the largest property 
managers in the country. The Real 
Property Services Branch of Public Works 
and Government Services Canada is the 
primary landlord for a significant 
proportion of the government’s real 
property holdings, the buildings in the 
Parliamentary Precinct and most of the 
many office buildings used by government 
departments and agencies across the 
country. As the landlord, the Department 
traditionally has provided a wide variety 
of real property services to its tenant 
departments, ranging from cleaning and 
fit-up to advice on how to make the best 
use of real property resources. It is also 
responsible for the ongoing care and 
maintenance of the assets. In all, the 
Department provides office space for 
some 160,000 public servants in about 
2,500 facilities across Canada. Its 
inventory amounts to about 6 million 
square metres of Crown-owned and leased 
space — from office buildings, 


laboratories and warehouses to the 
Parliament buildings and other “national 
treasures”. It also administers 3,000 leases 
representing annual rent of $550 million. 
This makes the Department one of the 
largest property managers in the country. 


18.12 Having defined its role during 
Program Review, the Branch undertook a 
study to identify its core services and 
activities — those it needed to retain to 
realize its vision of becoming the 
government’s strategic real property 
advisor and to fulfil its mandated role. At 
the same time, it would identify the 
services and activities that fulfilled no 
special public purpose and therefore could 
be carried out by the private sector. 


18.13. This “Core/Non-core” study 
generally classified as core activities those 
that dealt with issues of mandate, program 
management and strategy, quality 
assurance and risk management. 
Typically, the operational aspects of 
service delivery were identified as 
non-core and became candidates for 
contracting out. 


18.14 Asaresult of its study, the 
Department concluded that the property 
management services it was then 
providing to the government were 
non-core activities and that the private 
sector might be able to deliver them more 
cost-effectively. The types of service to be 
contracted out included cleaning, 
groundskeeping, mechanical maintenance 
and operation of building systems, and 
preparation of annual building 
management plans. The study further 
concluded that there was no strategic or 
public purpose reason why the 
government should continue to deliver 
these types of services itself. 


18.15 From these conclusions flowed 
the decision to contract out property 
management services and to negotiate the 
possibility of transferring to the successful 
bidders the PWGSC employees then 
delivering those services. 


The Department 
concluded that the 
private sector might be 
able to deliver 
property management 
services more 
cost-effectively. 
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The Department 
wanted to focus more 
on its advisory role 
and to save 10 percent 
on its property 
management costs. 


Exhibit 18.1 


Milestones in Contracting Out 
Property Management 


Source: PWGSC 


The decision to contract out 


18.16  Inrecent years, several large 
companies in Canada have contracted out 
their property management services, as 
have some Crown corporations such as 
Canada Post Corporation and the 
Canadian Broadcasting Corporation. The 
real property industry is evolving and 
developing the capacity to meet the needs 
of such large clients. 


18.17 The main objectives for 
contracting out property management 
services usually are to save money, 
concentrate on core activities and get 
access to first-class property management 
capabilities. However, contracting out an 
operation does not mean abandoning 
responsibility for it. Although the 
government has contracted for property 
management services, it retains 
responsibility for their funding and is 
accountable for them to building tenants 
and to Parliament. It is still the 
government that decides on the quantity 
and quality of a service to be provided, 
and it remains accountable for both 
aspects of that service. 


18.18 Inits November 1995 Business 
Plan, the Department stressed its intent to 


change the way its services are delivered. 
It wanted to focus more on its advisory 
role and to provide a full range of real 
property services more affordably and 
efficiently. In particular, it set a target for 
the AFD initiative to save 10 percent on 
its property management costs while 
providing at least the same level of 
service. The Branch established a number 
of principles to guide management in 
implementing the AFD, to ensure that: 


e employees would be protected to the 
greatest extent possible; 


e transferred assets would be 
maintained in good condition; and 


e it would remain the landlord of all 
the assets and continue to exercise all 
responsibilities inherent in that role. 


18.19 Once the Branch had decided to 
proceed with AFD in the fall of 1996, the 
timing of the process was driven by the 
need to complete the transfer of affected 
employees to the successful bidder before 
the terms of the government-wide 
programs for work force adjustment 
expired in March 1998. Exhibit 18.1 
shows key milestones in the 
contracting-out process. For a transaction 
of this size and complexity, this meant 
that a great deal had to be done ina 


Date Event 
31 Oct. 1996 Treasury Board informed of decision to contract out 
21 Feb. 1997 Request for Qualification issued 
8 Jul. 1997 Request for Proposal issued 
17 Sept. 1997 Proposals due 
17 Sept. 1997 to 10 Dec. 1997 Evaluation of proposals 
MEDEe 9977 Treasury Board approval of contracts 
12 Dec. 1997 Letters of intent to enter into contracts 
13 Dec. 1997 to 17 Mar. 1998 Negotiation of final wording of contracts 
18 Mar. 1998 Contracts signed 
19 Mar. 1998 Job offers made and lay-off notices given 
19 May 1998 Job offer acceptances due 
28 May 1998 Affected employees join the contractor 
28 May 1998 Handover of operations from PWGSC to contractor 
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relatively short time frame. Further, 
ensuring a fair and competitive process 
meant that the time allowed for certain of 
these activities, such as preparation and 
evaluation of tenders, could not be 
compressed. 


Buildings included in the AFD initiative 


18.20 Originally, the Department’s 
intent was to include in the initiative all 
buildings owned by the Crown that are 
under PWGSC’s custodianship, except 
those with unique and special 
requirements such as the Parliamentary 
Precinct, and buildings with security 
requirements. In some cases, however, 
client departments requested that the 
Department continue to deliver property 
management services directly, because the 
AFD initiative had no track record. The 
Department chose to accede to those 
requests and excluded the buildings from 
the first round of contracts. It told us that 
once it gains more experience with the 
AFD initiative, it will consider expanding 
it to include more buildings. Premises 
leased by the Department were also 
excluded from the initiative, since 
property services for them are already 
delivered by their private sector landlords. 


18.21 In structuring the contracting 
process, available options ranged from a 
separate contract for each building to one 
very large national contract for all of 
them. Ultimately the Department settled 
on a middle course. It broke the eligible 
properties into contracting packages, or 
building portfolios, each of which became 
the subject of a separate contract. 


18.22 Of the approximately 

487 buildings with about 3.5 million 
square metres of space that are owned by 
the Crown and are under PWGSC’s 
custodianship, more than 100 were 
excluded from the AFD initiative. The 
remaining 377 buildings were grouped by 
geographical area into 18 large portfolios. 
Management considered that large 
portfolios were necessary both to realize 
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economies of scale and to reduce as much 
as possible the costs of the departmental 
infrastructure and the associated corporate 
and administrative services. 


18.23. Since PWGSC had been involved 
with the provinces in “efficiency of the 
federation” initiatives since the early 
1990s, it first invited provincial and 
territorial governments to advise it if they 
were interested in providing the services. 
They showed interest in 5 of the 

18 portfolios. The remaining 13 portfolios 
were offered to the private sector in an 
open, competitive bidding process. In all, 
they consisted of 300 buildings with about 
2.3 million square meters of office space 
(see Exhibit 18.2). The same contractor 
was the successful bidder on each of the 
13 contracts. 


Scope of contracts 


18.24 The scope of AFD contracts for 
each portfolio included property 
management, project delivery, retail 
letting and tenant project services. The 
activities in each service category are 
shown in Exhibit 18.3. 


Our audit covered the 
13 contracts the 
Department signed 
with a private sector 
contractor. 


18.25 The Department’s 1998-99 gross 
expenditures related to federal 
accommodations amounted to about 
$1.33 billion, of which around 

$743 million represented rentals and 
grants to municipalities in lieu of taxes, 
and $230 million represented major 
capital expenditures (over $200,000). Of 
the remaining $357 million budgeted for 
the management and maintenance of 
Crown-owned buildings, about 

$211 million was budgeted for the 377 
buildings included in the AFD initiative. 


Focus of the audit 


18.26 Our audit covered the 13 
contracts the Department signed with a 
private sector contractor. We did not look 
at the five portfolios offered to provincial 
and territorial governments because not all 
of the arrangements had been finalized at 
the time of our audit. 
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Exhibit 18.2 


Building Portfolios Offered 
Under the AFD Initiative 


Source: PWGSC 


18.27 Our examination of the 
contracting process sought to determine 
whether the procedures that were followed 
were adequate to protect the interests of 
the government and the taxpayers and, in 
particular, whether: 


e the procurement process was 
conducted in a manner consistent with the 
objectives established for the AFD, and 
with the government’s rules and 
guidelines on procurement; 


e the resulting contract was, in its 
significant aspects, consistent with the 
successful proposal; 


e the transition to the contract was 
managed efficiently and in accordance 
with the contract; 


e the Department has established the 
procedures necessary to manage its 
relationship with the contractor; and 


e the Department has adequately 
ensured environmental protection and 
sustainable development in the AFD 
initiative. 

18.28 We did not examine the merits of 
the original decision to contract out 
property management services. 
Additionally, the timing of our audit work 
was such that an examination of the 
financial and operational results of the 
AFD for the initial operational period 
ending 31 March 1999 was not practical. 
The field work for this audit was 
completed | April 1999. 


Portfolios offered to provincial Number of Floor Area 
and territorial governments Buildings (square metres) 
British Columbia (except Vancouver) 30 206,802 
Northwest Territories * 20 13,911 
Prince Edward Island * i 31,046 
Saskatchewan 13 30,084 
Yukon Territory * 7 15,902 
Sub-total Wi 297,745 
Portfolios offered to the private sector 

Alberta 10 IBY Poul 
Manitoba 2 121,188 
Ontario (except Toronto) 58 234,191 
National Capital Area — #1 13 345,437 
National Capital Area — #2 4 469,950 
National Capital Area — #3 23 253,425 
New Brunswick 32) 59,316 
Newfoundland 42 65,913 
Nova Scotia 28 69,471 
Quebec — East 3] 118,433 
Quebec — West 30 257,680 
Toronto 8 170,165 
Vancouver 9 58,731 
Sub-total 300 2,299,137 
Total 377 2,596,882 


* An agreement has not yet been reached. The Department continues to maintain these portfolios. 
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18.29 We did not audit the prime 
contractor, Brookfield Lepage Johnson 
Controls Facility Management 
Services(BLJC), and nothing in this 
chapter should be construed as a comment 
about it. We were concerned solely with 
the actions of the government. 


18.30 Further details on our audit 
objectives, criteria and approach are 
presented in About the Audit at the end 
of the chapter. 


Observations and 
Recommendations 


The Contracting Process 


Solicitation, evaluation and award 
phases were well managed 


18.31 Key stakeholders and, in 
particular, real property companies and 
industry associations were consulted 
before and during the process to solicit 
their views and to keep them up-to-date 
on the progress of the AFD initiative. The 
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contracting for the AFD initiative was set 
up using a two-step tendering process for 
each portfolio. The first step, in February 
1997, involved posting the Request for 
Qualifications (RFQ) document on the 
Department’s electronic Open Bidding 
System. The RFQ was intended to provide 
the industry with notice of the 
Department’s intent, provide potential 
suppliers with a statement of the 
qualifications they would need to bid for 
the contract(s), and permit the Department 
to gauge the extent of potential 
competition. Issuing the RFQ also meant 
that potential suppliers who might not 
possess all of the desired qualifications 
had the opportunity to strengthen their 
organization, or to form partnerships with 
others and create a capability to fully meet 
the government’s requirements. The RFQ 
elicited responses from 28 potential 
suppliers. 


18.32 The second step was to issue the 
Request for Proposals (RFP) in July 1997. 
It drew at least three bids on each of the 
13 portfolios (in all, 62 proposals for the 
13 portfolios from 14 different service 


Exhibit 18.3 


Scope of AFD Contracts for Each Portfolio 


Property Management Project Delivery Retail Letting Services Tenant Delivery 
Services | Services (where applicable) Projects 
—— 

e Preparation of building e Management and delivery e Management and e All projects requested by 

and portfolio plans of construction projects administration of tenants (funded directly by 
° Day-to-day operations (repairs, upgrade, fit-up third-party leases tenants and not included in 

and space optimization the price of the contracts; 

* Energy management projects up to a value of estimated annual amount: 
e Cleaning $200,000) $10.8 million nationally) 
e Maintenance 
e Landscaping 
e Environmental protection | 
e Health and safety 

requirements 
e Emergency preparedness 
e Property management | 

administration 

— SS = — = 


Source: PWGSC 
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A significant feature of 
the contracting 
process was the 
Department’s use of 
“fairness monitors’. 


providers) with the exception of the 
Alberta portfolio, for which two bids were 
received. 


18.33. The Department established a 
multi-disciplinary team of experts to 
evaluate the bids. They used a point-rating 
system that took into account key factors 
such as the proposed approach to service 
delivery (with 35 percent of possible 
points), quality and quantity of job offers 
to existing departmental employees 

(35 percent), qualifications (20 percent), 
and price and fees quoted (10 percent). 
The relative weight to be given to each 
area was known to the bidders and was 
intended to reflect the Department’s 
priorities and objectives in entering into 
the contracts. 


18.34 To discourage unbalanced bids or 
attempts to “buy” the contract, the 
Department structured its evaluation 
methodology to emphasize technical 
superiority over price. Bids were required 
to meet a pricing target established by the 
Department. This target was based on the 
amount that the Department estimated it 
would have spent to provide these services 
itself, less 10 percent. However, bids 
would gain few points for being lower 
than the target amount. Consistent with 
the guiding principles established for the 
AFD, the weighting of factors strongly 
encouraged bidders to present attractive 
offers to the Department’s employees. 


18.35 On 12 December 1997, following 
the completion of the evaluation process, 
the Minister responsible for PWGSC 
announced that BLJC had been selected as 
the sole contractor for all the contracts, 
since it had submitted a superior bid for 
each of the 13 portfolios. 


Fairness monitoring provided assurance 


18.36 A significant feature of the 
contracting process was the Department’s 
use of “fairness monitors”. The first 
independent monitor provided advice and 
guidance to ensure the integrity of the 
overall AFD process, and the second 


18—12 


monitor carried out “fairness” reviews at 
the solicitation, evaluation and award 
phases of the contract. The Department 
thereby received a high level of assurance 
that this large and potentially 
controversial project was conducted fairly. 


18.37 As the scope of the fairness 
review work overlapped substantially with 
our audit scope, we sought to determine 
whether and to what extent we could rely 
on that work and reduce our own testing. 
Based on interviews with the two monitors 
and a review of their work and relevant 
documentation, we found that we could 
rely on their opinions of the fairness of the 
process (see Exhibit 18.4 for their 
opinions and a brief description of the 
scope of their work). 


18.38 In our view, the independent 
assurance provided by the “fairness 
monitoring” process became particularly 
important when the contracts for all 13 of 
the portfolios up for bid were awarded to a 
single bidder. Such a process involves 
costs and may not be appropriate for all 
contracts. For very large and possibly 
controversial contracts, however, 1t may 
be worth the cost to be able to provide all 
parties — management, ministers, 
Parliament, industry and taxpayers — 
with timely and independent assurance 
that the contracting process used was fair 
and procedurally correct. 


Specifics of the contracts 


18.39 = All 13 contracts contain the same 
basic standard terms and conditions. The 
contracts are for a term of two years and 
10 months (28 May 1998 to 31 March 
2001). The Crown has the option of 
extending the term by up to four years. 
Therefore, the contracts could extend up 
to six years and 10 months before being 
tendered again. 


18.40 The pricing method used is “cost 
plus fee”. This means that PWGSC 
reimburses the contractor for only those 
eligible costs it incurs in managing the 
buildings, plus the related management 
fee. Eligible costs include, among other 


Report of the Auditor General of Canada — September 1999 


things, disbursements the contractor 
makes to deliver services, and labour costs 
for positions up to the level of property 
manager or equivalent. The management 
fee is a predetermined percentage of the 
costs incurred. This percentage, between 

5 percent and 6.8 percent, varies in each 
portfolio and according to the type of cost 
involved. 


18.41 Overall, the Crown’s annual 
financial obligation under each of the 
contracts 1s subject to a predetermined 
ceiling, established by the contractor’s 
initial bid. This means that the contractor 
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must deliver all services described in the 
statement of work — including property 
management, project delivery and retail 
letting services — for an amount equal to 
or less than the Crown’s annual financial 
obligation. The sum of that annual 
financial obligation for the term of the 
contract (two years and 10 months) 
constitutes the estimated contract price. 
Exhibit 18.5 shows the bids submitted by 
the successful contractor. 


18.42 The amount of the Crown’s total 
annual financial obligation includes an 
estimated amount between $8 million and 


Exhibit 18.4 


Fairness Monitoring - Opinions on the Fairness of the Contracting Process 


Monitor 1 


| Monitor 2 


Scope of inquiry ° 


criteria to be used 


contract negotiations 


The development of the Request for Proposal and e 
the Statement of Work and Model contract 


e The development of the evaluation process and the ° 


e The bid evaluation and selection including the 
identification of issues and the resolution thereof ° 


e The monitoring of conference calls during the 


Reviewed the Request for Proposal and the 
Request for Qualification documents. 


Reviewed the evaluation plans and scoring 
methodologies/plans developed for the evaluation 
of the RFP and RFQ. 


Reviewed the work and approach used by the team 
members responsible for conducting the 
evaluation of the RFP and RFQ. 


e Attended or reviewed tape recordings of all 
debriefing sessions provided to bidders who 
submitted information at the RFQ stage of the bid 
evaluation process. 


e Participated in the resolution of all significant 
fairness issues raised during the course of the 
evaluation of the RFQ and RFP. 


e Reviewed the evaluation scoring of the successful 
bidder. 
e Attended the debriefings of unsuccessful bidders. 


e Reviewed the contract between PWGSC and the 
successful bidder. 


Conclusion 


“In our opinion, the overall AFD process was 


“Nothing has come to our attention that leads us to 


conducted with integrity, including equity, 
openness and fairness of the contracting process 
enabling consistent reasonable and fair treatment of 
prospective service providers.” 


believe that the bid evaluation/selection activities 
were not conducted with integrity, openness and 
fairness. No issues have arisen which would lead us 
to believe that all prospective services providers were 
not treated in a consistent, reasonable and fair manner 
and that the bid evaluation/selection activities 
undertaken were not in accordance with those 
described in the RFQ and RFP.” 


Source: The fairness monitors’ opinion letters. 
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An innovative feature 
of the contract is the 
effort to refocus 
contract management 
on performance rather 
than process. 


$9 million in management fees for the 
contractor. It is important to note that 
these fees do not represent the contractor’s 
profit, but rather are its gross revenue for 
these contracts. From these fees the 
contractor in turn must cover its operating 
costs and the costs of administering the 
contracts. 


Innovative fee structure and 
performance monitoring 


18.43 An innovative feature of the 
contract is the effort to refocus contract 
management on performance rather than 
process. Accordingly, the contract 
specifies that during the contract year, the 
contractor will be reimbursed for eligible 
costs incurred and for 85 percent of the 
related management fee. The remaining 
15 percent is not to be paid until year-end. 
This year-end payment (a performance 
fee) depends on the extent to which the 
contractor has performed at levels agreed 
upon at the beginning of the year. The 
contractor can earn an additional 5 percent 


of the measured performance areas for 
that service category. 


18.44 To implement this provision, the 
Department has developed and written 
into the contract a comprehensive 
framework of 32 performance indicators 
against which to monitor and measure 
BLJC’s performance. They are grouped 
into three categories: asset integrity, 
financial, and satisfaction (see 

Exhibit 18.6). 


18.45 The use of key performance 
indicators places emphasis on 
performance rather than process, by 
linking fees earned to performance. 
However, this requires that both parties 
define and agree in advance on the 
performance expected. 


Human Resource Issues 


18.46 Human resource issues lie at the 
heart of the AFD initiative, for two 
reasons. First, the Department had 
established as a guiding principle for the 


initiative the greatest possible protection 
of the position of its affected employees. 
Second, doing so would, in the 


of the total management fee (a service 
excellence fee) in each of three service 
categories if its performance excels in all 


Exhibit 18.5 Bids Expressed on a 
Bids for the Term of 12-Month Basis 
Bids of the Successful Contractor 2 Years 10 Months (Crown’s Maximum 
(Estimated Annual Financial 
Building Portfolios Contract Price) Obligation) 
1. Vancouver 15,925:132 5,590,389 
2. Alberta 12,619,008 4,428,193 
3. Manitoba 19,340,741 6,791,080 
4. Ontario (except Toronto) 46,832,766 16,445,706 
5. Toronto 38,424,816 13,481,227 
6. National Capital Region — #1 69,198,842 24,222,089 
7. National Capital Region — #2 115,113,950 40,380,167 
8. National Capital Region — #3 59,384,652 20,826,462 
9. Quebec — West 49,119,319 17,234,117 
10. New Brunswick 13,035,578 4,572,604 
11. Nova Scotia 13,136,591 4,608,725 
12. Newfoundland 11,425,536 4,006,843 
13. Quebec — East 22,870,061 8,019,448 
Source: PWGSC Total 486,426,992 170,607,050 
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Department’s view, require that the 
transaction take place under the provisions 
of the government’s work force 
adjustment program set up to help 
employees deal with the changes that 
Program Review had generated. Key 
provisions of the program were to expire 
on 31 March 1998 and so by that date the 
Department would have to resolve all of 
the key issues surrounding the 
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employment status of those affected. This 
established hard deadlines for certain 
events that became critical drivers of 
many decisions the Department made in 
the transition phase of the contract, 
discussed later in this chapter. 


18.47 _In the RFP, the Department 
included minimum requirements that 
bidders had to meet in the job offers they 


Exhibit 18.6 


Performance Indicator Framework to Monitor and Measure Contractor’s Performance 


7 
Maximum Maximum Maximum 
Cc Property Management Possible Project Delivery Possible Rental Letting Possible 
ategory Services Score Services Score Services Score 
= == 
Asset integrity Critical events 150 Scope 250 
Energy consumption 100 Quality 750 
Integrity assessment 400** N/A 
Quality control inspections 350 
Subtotal | 1000 1000 
Financial PMS baseline 600 Portfolio project baseline 250 _||Rental rates 250 
Forecasting of year end 
actuals 400 Estimates 250 _||Revenues 250 
Value for Money 250 _||Account >15 days 250 
Forecasting of year end 250 ||Occupancy levels 250 
Subtotal | 1000 1000 1000 
=F + 
Satisfaction Professionalism 100 Project plan 300 ||Letting Approach 500 
Progress SDS 100 Project delivery 300 || Letting operations 500 
Effective sub-contracting 100 Delivery against plan 400 
Information and reporting 100 
Service calls 200* 
(Maintenance) 
Security services 200 
Managing commercial 
letting 100 
Commercial letting 
revenue 100 
| Service calls — 400* 
Operations 
| Responsiveness 400* 
Tenant surveys 200* | 
Subtotal 2000 1000 | 1000 | 
— - ————— — — — — ——- — = - aa | 
~ TOTAL | | 4000 3000 | 2000 
e = —— = — J = a = = = rs i a] eS =| 


* Performance indicators were not assessed during 1998-99. 


** Baseline assessment done during the year, year-end assessment not done in 1998-99. Source: PWGSC 
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The Department had 
established as a 
guiding principle for 
the initiative the 
greatest possible 
protection of its 
affected employees. 


The necessary steps 
were taken to ensure a 
proper transition from 
management by the 
Department to 
management by the 
contractor upon the 
handover. 


submitted with their bid proposals. The 
Department gave the human resources 
component of the proposals a weighting of 
35 percent when it evaluated the 


‘submitted bids. It found that BLJC’s 


proposals exceeded its minimum job offer 
requirements. 


18.48 In its proposal, BLJC committed 
itself to offering jobs to almost all 
available operational employees, at their 
current pay and with a minimum 
three-year employment guarantee or for as 
long as BLJC keeps the contracts. In early 
1997, about 600 departmental employees 
were involved in managing and operating 
the buildings in the 13 portfolios offered 
to the private sector. In the ensuing 
months, about 100 of those employees 
found a job either elsewhere in the 
government or outside the government. As 
a result, only 500 employees were 
available in March 1998 when BLJC 
made its job offers; 465 accepted and 
joined BLJC on 28 May 1998. 


18.49 Employees who joined BLJC 
received cash payments. We found that 
the Department’s compensation payments 
to these employees complied with the 
Treasury Board’s work force adjustment 
programs. We noted that the method it 
used to determine the final number of 
persons available for job offers was 
appropriate, and that BLJC respected the 
job offer commitments it had made in its 
proposals. 


Transition Period 


Tasks still remain to be completed 


18.50 Between the contract award in 
late 1997 and the date when the services 
were handed over to the contractor 

(28 May 1998) there was a transition 
period in which a significant number of 
tasks had to be completed. As already 
noted, the tasks that involved the 
transition of employees were 
accomplished on time. Further, the 
necessary steps were taken to ensure a 


proper transition from management by 
PWGSC to management by the contractor 
upon the handover. By all reports, the 
handover went smoothly. 


18.51 In the transition period, however, 
the contractor did not deal with several 
matters related to operational budgets, 
building condition, and the establishment 
of baseline measures for the key 
performance indicators. The contractor 
was to have addressed these issues before 
the contract was signed. They were 
rescheduled to the handover date, 28 May 
1998, but at that date most had still not 
been completed. The Department advised 
us that there had been agreement between 
the parties to defer these issues until other 
priorities were dealt with. 


18.52 The building management plans 
containing budget baselines and projects 
are a key control for the Department. 
Once reviewed and approved by the 
Department, these plans constitute the 
basis on which the contractor 1s to manage 
the buildings for the planning year in 
question. They define the ceiling on 
eligible expenditures and the extent and 
scope of approved minor projects for the 
structures. They were to have been 
prepared by the contractor and reviewed 
and approved before the handover. They 
were not completed and approved until 
well into the first year of the contract. 


18.53 Before the handover, the 
contractor was also to have formally 
inspected all buildings. The inspection 
results were to have formed a key baseline 
for agreement between the contractor and 
the Department on the initial condition of 
the buildings, as well as the baseline for 
measuring asset integrity performance in 
the first year of the contract. The 
contractor did not do this work, and in 
summer and fall of 1998 the Department 
carried out the inspections at its own 
expense. In these circumstances, it was 
very important that the Department 
receive the contractor’s formal 
concurrence in the assessments of building 
condition. In some of the contracts there is 
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written evidence of concurrence, but in 
others the Department does not have such 
formal assurance from the contractor. 


18.54 Establishing key performance 
indicators required that both parties define 
performance. This proved to be much 
more complex than anticipated, and most 
baselines for assessing performance were 
finalized only well into the first year. One 
consequence is that for much of the year, 
PWGSC had no indicators against which 
to assess the contractor’s performance. 
Some of the performance indicators will 
not be used at all this year, since related 
baselines for measuring performance have 
not yet been established (see 

Exhibit 18.6). More than half (on a 
weighted basis) of the indicators of 
customer Satisfaction with property 
management services will not be used or 
usable this year. As noted above, the core 
measure of asset integrity is also affected. 
As with the building condition reports, 
evidence of the contractor’s agreement on 
the basis for measuring performance is 
inconsistent among the contracts. Some 
regions had full, written documentation 
while others were able to assure us only of 
oral agreements. 


18.55 We note that the contracts 
provide for paying the contractor 
“performance fees” and “service 
excellence fees”. For the purpose of 
determining its eligibility for these fees, 
the contractor’s performance is to be 
measured according to the key 
performance indicators. Given that some 
indicators have yet to be established, the 
contractor could be paid performance fees 
and possibly excellence fees this year for 
results that may have been less than were 
required. Further, this could lead to 
problems and disputes over payments. 


18.56 The delays in completing all of 
the transition tasks arose partly because 
priority was given to meeting the human 
resources deadline; the contractor had to 
make the job offers to affected employees 
on or before 31 March 1998. In a project 
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of this scope and complexity and given 
the fixed time constraints, we did not 
expect that everything could have been 
completed in time. We did expect, 
however, that the Department would have 
established a process to ensure the timely 
and controlled completion of the tasks that 
remained to be done. The two parties 
continue to work on these items, but the 
Department has not formally notified the 
contractor of the work that is incomplete, 
nor has it developed and agreed with the 
contractor on a plan containing target 
dates and respective accountabilities for 
resolving these issues. 


18.57. The Department should develop 
a schedule of outstanding contractual 
items, and develop an action plan with 
the contractor showing when each item 
will be completed and which party will 
be responsible for completing it. 


We did not expect that 
everything could have 
been completed in 
time. We did expect, 
however, that the 
Department would 
have established a 
process to ensure the 
timely and controlled 
completion of the 
tasks that remained to 
be done. 


Department’s response: Public Works and 
Government Services Canada accepts the 
recommendation. The Department has 
been actively working with its service 
provider to put in place the outstanding 
deliverables and will develop an action 
plan with target dates. 


Expected Savings 


18.58 One of the Department’s 
objectives for the Alternative Forms of 
Delivery (AFD) initiative was to achieve a 
10 percent annual saving on its property 
management costs. The Department told 
us that this target was based on the 
experience of other organizations and on 
discussions with industry representatives. 
It believes that the contractor has 
sufficient opportunity to achieve the 
savings of 10 percent by minimizing 
management costs, particularly in the 
areas of human resources, contracting and 
finance — in other words, by reducing 
administrative costs rather than cutting 
operating and maintenance costs to a point 
that might affect client satisfaction and 
compromise the operation and 
maintenance of buildings. 
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We could not 
determine whether or 
not the contracts will 
yield the expected 

10 percent savings. 
We did find that the 
contracts contain 
control mechanisms to 
facilitate savings. 


Determining the cost base 


18.59 The Department used its 
projected costs for 1998-99 as a baseline 
for measuring savings. From this baseline, 
it deducted its estimated costs of 
managing the contracts and the estimated 
10 percent savings (in total, about 

$20 million) and arrived at the target cost 
for each portfolio. The bidders were 
required to offer the specified services for 
an amount no greater than the target cost. 


18.60 The challenge was then to make 
sure that the baselines would reflect the 
costs actually incurred by the Department. 
The baselines were calculated in 1997, 
using estimates based on actual data for 
1995—96 and year-end forecasts for 
1996-97. Using the actual costs for 
1997-98, we made our own calculation of 
the costs of maintaining the buildings 
included in the AFD. We could not 
reconcile our baselines and those of the 
Department with sufficient precision to 
conclude whether the baselines were 
accurate. This is due in part to material 
errors made by the Department that 
overstated the baselines used in the 
Request for Proposals issued for some 
major portfolios. However, the 
Department discovered its errors by 
applying its own control procedures and 
subsequently negotiated a number of 
amendments to the contracts. Although 
this lowered the cost baselines in those 
cases, they were still higher than they 
should have been. As a result, the 
government will spend more money on 
those portfolios than it might have spent 
had the original cost baselines been 
calculated accurately. The Department has 
informed us that the additional funds have 
been used to do maintenance and repair 
work that would otherwise have been 
deferred to future years. 


18.61 A review of actual results for the 
first year of operation showed that the cost 
baselines established for property 
management services were exceeded in 
some portfolios. The total ceilings on 


those portfolios were not exceeded, 
however, because less money was spent 
on the project delivery services. 
Nonetheless, the Department needs to 
determine why the baselines were 
exceeded if it is to ensure that cost 
reductions are not achieved at the expense 
of repair and/or upgrade projects; this 
could have an impact on asset integrity in 
the longer term. 


18.62 The inventory of buildings is 
changing constantly, and amendments are 
needed to reflect those changes. The 
contracts did not contain a structured 
protocol to determine which costs are to 
be deleted (or added) in the event that a 
building is removed from (or added to) the 
portfolio. Amendments are negotiated 
case by case by the respective contracting 
authorities. 


Obtaining expected quality and quantity 
of services 


18.63 The concept of a saving implies 
getting the same or better service at lower 
cost. To ensure that it would get the 
quality and quantity of services specified 
in the contracts, the Branch designed a 
comprehensive performance evaluation 
system. In the first year of the contracts, 
however, the Department has not been 
able to use many of the performance 
indicators, including some dealing with 
client satisfaction and asset integrity (see 
paragraph 18.54). Other than anecdotal 
information, the Department does not 
have enough data to measure customer 
satisfaction. This will affect the quality 
and completeness of the contractor’s 
performance evaluation by the 
Department. The Department is currently 
finalizing those performance indicators 
and believes that the performance 
evaluation system will be fully operational 
for the second year of the contract. 


18.64 Given the evidence we had, 
however, we could not determine whether 
or not the contracts will yield the expected 
10 percent savings while maintaining at 
least the same level of service. Nor could 
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we determine at this time whether assets 
are likely to be maintained in a condition 
at least as good as when the Department 
was providing the services. 


18.65 We did find that the contracts 
contain control mechanisms to facilitate 
savings. The extent to which departmental 
personnel monitor the contractor’s 
performance will determine whether the 
initiative is actually yielding savings. The 
Department will need to scrutinize annual 
building management plans and 
operational baselines and to monitor 
actual performance against them. It will 
also need to assess the contractor’s cost 
effectiveness in delivering tenant project 
services. Only by doing so will the 
Department be able to ensure that it is 
continuously getting the value it expects 
for the money it spends. The Department 
will need to arrive at an adequate balance 
between ensuring that the contractor is 
fulfilling the terms of the contracts and 
ensuring that its own cost of managing the 
contracts does not offset any savings. 


18.66 The contracts incorporate 
features that should foster co-operation 
and promote innovative business 
processes — both of which should lead to 
lower costs. At any time during the term 
of the contracts, the contractor may 
submit proposals under the Management 
Value Incentive Program. This is a 
program that provides incentives for the 
contractor to develop proposals that will 
produce savings. If PWGSC accepts a 
proposal, it becomes a project and the 
contractor and the Crown share any 
savings that result. If properly used and 
controlled, this incentive program could 
lead to lower costs in the future. 


Environmental Issues in the 
Contract 


18.67 The extent to which the 
management of the government’s real 
property assets reflects the Department’s 
sustainable development strategy (SDS) 
directly affects the environment. The 


contracts made appropriate provision for 
the contractor to support the Department’s 
environmental activities and to take 
necessary actions to adhere to the SDS. 
Departmental asset managers will 
regularly monitor the contractor’s 
performance and will measure it annually. 
How well BLJC performs on 
environmental matters directly affects the 
fees it will receive under the performance 
plan. 


18.68 — All of the buildings initially 
covered by the AFD initiative were 
subject to environmental reviews by the 
Department, and many priorities for action 
were identified. The contractor’s initial 
portfolio management plans that we 
reviewed did not show a clear link to these 
assessments. Furthermore, at the end of 
our field work, BLJC was finalizing 
detailed Environment Protection and 
Conservation Plans for all portfolios. 
These plans are essential both to support 
the Department’s environmental activities 
and to adhere to the sustainable 
development strategy. They are also 
required under the contract. 


Conclusion 


18.69 Our examination of the 
contracting process, from the Request for 
Qualifications and the Request for 
Proposals to the evaluation of bids and 
awarding of the contract, led us to 
conclude that in all material respects it 
was carried out in accordance with the 
regulations and rules for government 
contracting. Further, nothing came to our 
attention during the course of our audit 
that would lead us to doubt the 
conclusions of the fairness monitors that 
the process was conducted in a fair and 
transparent manner. 


18.70 We examined the solicitation and 
evaluation process and the terms of the 
contracts themselves in relation to the 
objectives established for the contracting 
out of property management. The design 
of the process was, except for errors in 


Nothing came to our 
attention that would 
lead us to doubt the 
conclusions of the 
fairness monitors that 
the process was fair 
and transparent. 
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costing, consistent with the objectives and 
priorities that the Department had 
established. The contracts as signed on 

18 March 1998 reflected, in all significant 
aspects, the proposals that the winning 
contractor had submitted on 15 September 
1997. The contracts were well designed 
and the handover to the contractor went 
smoothly. Employees were treated in 
accordance with the Treasury Board’s 
work force adjustment programs. 


18.71 The monitoring and performance 
management procedures needed to track 
whether the contractor is fulfilling all its 
obligations were not in place on time. As 
a result, the Department will not be able 
to base its 1998-99 assessment of the 
contractor’s performance on a complete 
set of performance indicators. 


18.72 It is too early to assess whether 
savings expected by the Department from 
this initiative will be achieved. 
Calculations of cost bases were not 
verified independently to provide 
management with additional assurance 
that they were reliable and accurate. 
However, mechanisms that are conducive 
to achieving savings were incorporated in 
the contracts. 


Some lessons learned 


18.73 In large and complex transactions 
of this kind, the use of “fairness” monitors 
can provide assurance that the process is 
transparent and fair. Additional assurance 
on whether financial objectives are 


achieved can be obtained by subjecting 
the calculation of cost bases to 
independent verification. 


18.74 Contracting out transactions of 
this magnitude is a complex process that 
takes a lot of time. Realistic timelines 
need to be set. However, it cannot be 
expected that everything will be ready on 
the start date, and management cannot 
wait for everything to be in place before 
going ahead. This needs to be recognized 
up front and contingency plans prepared 
early in the process to provide a road map 
for both parties to deal with any 
uncompleted tasks. 


18.75 Having a good system for 
documenting all decisions and keeping 
records of all agreements between parties 
is an important element of a good 
accountability framework. 


18.76 The transfer of service delivery 
responsibilities to a service provider in the 
private sector represents a major shift for 
departmental personnel, who must learn 
how to oversee the work effectively and 
efficiently rather than doing the work 
themselves. 


18.77 In the case of the AFD initiative, 
the Department conducted extensive 
consultations before entering into the 
process, established clear goals for the 
initiative and developed an accountability 
framework that defined respective roles 
and responsibilities. This strengthened the 
arrangement and increased the initiative’s 
chances of success. 
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About the Audit 


% 


Objectives 


Our objective was to examine the contracting out of property management services by PWGSC to determine 
whether procedures that were followed were adequate to protect the interests of the government and the 
taxpayers. We sought to determine whether the contracting process had been consistent with the objectives of 
the Alternative Forms of Delivery (AFD) initiative and had been open and transparent. We also wanted to 
determine that PWGSC had implemented an adequate accountability framework and controls to ensure that 
the contractor would comply with the terms and conditions of the contract. 


Scope 


The scope of this examination included the part of the AFD initiative related to the 13 contracts for the 
property management services that were signed with the same private sector service provider. It did not 
include similar agreements signed with other governmental jurisdictions. We have not examined the decision 
to contract out the delivery of property management services. Our audit consisted of interviews with 
departmental management and other key stakeholders as well as an examination of all contracts and other 
relevant documentation. 


We have not audited the activities of the bidders and consequently are not commenting on them. 


Approach 


We looked at the contracting process used to transfer the management and operation of federal buildings to 
the private sector contractor, and at the Department’s administration of the contract during the first 10 months 
following the transfer. This was because the first 10 months represented the contract’s first complete cycle. 
Furthermore, we have found in examining other large contracting projects that many of the patterns leading to 
the eventual success or failure of a contract are established in its early stages. However, the timing of our 
audit work was such that an examination of the financial and operational results of the ADF for the initial 
operational period ending 31 March 1999 was not practical. The field work for this audit was completed 

1 April 1999. 


Criteria 


Contracting-out process and handover. We expected that the contracting-out process would be managed in 
a manner consistent with the Department’s objectives for the AFD, and that a competitive bidding process 
would be used and conducted with integrity and openness. We expected that due diligence would be 
demonstrated to ensure a smooth handover and minimize risks. 


Financial objectives. We expected that the initiative would be designed and implemented in a way that could 
ensure the expected cost savings without affecting the quality of property management services provided and 
the integrity of the assets. 


Human resource management issues. We expected a uniform treatment of employees affected by the 
initiative, in accordance with the Treasury Board’s work force adjustment programs. 


Report of the Auditor General of Canada — September 1999 18-2 


Public Works and Government Services Canada — Alternative Forms 
of Delivery: Contracting for Property Management Services 


Monitoring and performance management. We expected that the Department would design and implement 
monitoring and performance management procedures to track whether the contractor is complying with the 
terms and conditions of the contract and is continuously delivering the specified services at the quality 
required and for the cost agreed to. 


Environment and Sustainable Development. We expected that the contracts would require compliance by 
the contractor with the Department’s sustainable development strategy and would take into account the 
latter’s responsibilities under the Canadian Environmental Assessment Act. 


Audit Team 


Assistant Auditor General: Shahid Minto 
Principal: Hugh McRoberts 
Director: Alain Boucher 


Eric Hellsten 
Rosemary Marenger 
Janet Hatt 

Holly Shipton 


For information, please contact Hugh McRoberts. 
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Main Points 


19 This audit sought to determine if four grant and contribution programs, through which over $1.3 billion 
was spent over the last three years, were well designed to help improve Canada’s innovation performance. We 
audited the following programs: 


e Industrial Research Assistance Program — National Research Council (NRC) 

e Research Partnerships Program — Natural Sciences and Engineering Research Council (NSERC) 
e Networks of Centres of Excellence — NSERC 

e Technology Partnerships Canada — Industry Canada 


19:2 We expected that these programs would be based on a sound understanding of innovation performance 
problems in the economy. We found that while there is a wealth of information on various aspects of innovation 
performance in the economy, it is unclear what this information means when considered together. Moreover, we 
found that management has not defined what specific innovation performance problems these programs are 
supposed to address, nor what specific results are expected from them toward promoting innovation. 


195 We also sought to determine if these programs were well managed and if management knew whether 
value for money was being achieved. We could not assure ourselves that many of the contributions under the 
Industrial Research Assistance Program and many of the grants under the Research Partnerships Program were 
properly supported. While the technological merit of the projects we examined had been well documented, there 
was often little explanation of the commercial or pre-commercial benefits expected from the projects, and of the 
need for government support. There are also important performance issues for which management has little 
information — in particular, on the commercial or pre-commercial results of funded projects. 


19.4 We concluded that due diligence had been exercised in the grants we audited under the Networks of 
Centres of Excellence program. We also concluded that the management of Technology Partnerships Canada 
(TPC) had exercised due diligence in making the contributions that we audited, with specific exceptions. TPC 
could make improvements in monitoring the progress and results of funded projects, and in reporting to 
Parliament on how it shares risks and returns with funding recipients. 


Background and other observations 


19.5 The government has made building a more innovative economy one of its policy goals. A number of 
recent government reports have referred to an “innovation gap”, meaning that Canada is not innovative enough 
compared with its main trading partners. These reports argue that weaker innovation performance lies at the heart 
of broader performance problems in the economy — particularly lower productivity in relation to the United 
States. 


19.6 However, our review of the issues suggests that the causes and effects of this gap are not straightforward. 
While innovation is undoubtedly an important factor in economic growth, assessing the actual innovation 

performance of the economy is a multifaceted challenge. Although a comprehensive assessment is still difficult to 
make, there is growing evidence that Canada’s performance lags behind that of its major competitors in a number 


Report of the Auditor General of Canada — September 1999 19-5 


of specific ways. It is reasonably clear that better innovation performance depends on more than just increased 
spending on research and development; it involves supporting research and development with the activities needed 
to embed new technologies in the economy. It seems equally clear that spending on research and development is 
not the only determinant of the Canadian economy’s rate of productivity growth, and may not be the most 
important one. 


19.7 Promoting innovation in the economy is one of the principal objectives of the Industry Portfolio, which is 
made up of the organizations for which the Minister of Industry is responsible. The programs we audited focus on 
supporting research and development and account for the bulk of the grants and contributions made by the 
Portfolio toward that objective: 


e The National Research Council delivers the Industrial Research Assistance Program, which helps 
small- and medium-sized enterprises develop and exploit technologies ($120 million in 1998-99). 


e The Natural Sciences and Engineering Research Council delivers the Research Partnerships Program 
with the objective of fostering interactions and partnerships between university researchers and other 
sectors in order to generate new knowledge and develop new expertise, and to transfer this new 
knowledge and expertise to Canadian-based organizations ($95 million in 1998-99). NSERC also 
delivers the Networks of Centres of Excellence program to improve Canada’s performance in science 
and technology, and to facilitate transfer of knowledge to those who can use it to advance Canada’s 
social and economic development ($47 million in 1998-99). 


e Technology Partnerships Canada is a special operating agency within Industry Canada. It is intended 
to promote the development and commercialization of innovative technologies that contribute to 
increasing economic growth and creating jobs and wealth ($250 million in 1998-99). 


The responses of Industry Canada, the National Research Council and the Natural Sciences and Engineering 
Research Council to our recommendations are included in the chapter. The two councils have agreed to act 
on all of our recommendations. Industry Canada has agreed to act on all but one of our recommendations. 
The Department has indicated that no additional measures are required to address our recommendation 
concerning the justification for the amount of its contributions. 
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Introduction 


19.8 Over the past three fiscal years, 
Industry Canada, the National Research 
Council and the Natural Sciences and 
Engineering Research Council spent over 
$1.3 billion in grants and contributions 
through the programs we audited. 
Promoting innovation in Canadian 
industry through support for research and 
development is one of the key roles of 
each of these programs. This chapter 
presents: 


e areview of the issues of innovation 
performance in the economy; and 


e the results of our audit of these 
organizations to determine whether 
management is exercising due diligence 
and achieving value for money in their 
investments in innovation projects. 


1939 In recent years, innovation in the 
economy has received increasing 
attention. Before we present the results of 
our audit, it is worth reviewing what 
innovation means, and how it relates to 
the issues of national innovation systems, 
the innovation gap, and other economic 
performance issues. 


19.10 What does innovation mean? 
There is probably no single definition of 
innovation that would meet with universal 
approval. Nevertheless, the newness of a 
product or process is clearly a necessary 
criterion; and innovation is a broader idea 
than invention alone, for it also includes 
the development, application and 
commercialization of the invention. 
Innovation, then, is not only the initial 
flash of inspiration or the scientific 
breakthrough, but the process of rendering 
the idea practically or commercially 
viable. The United States General 
Accounting Office, in a 1996 report to 
Congress, defined innovation as both 
invention and commercialization. While 
this definition limits the idea to mainly 
business settings, the implicit point seems 
to be that innovation, wherever and 
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however it occurs, means knowledge put 
to new use. Beyond this broad meaning, 
there are a number of views on how to 
further define or to categorize different 
models of innovation (see Exhibit 19.1). 


19.11 National and regional 
innovation systems. There is more to 
innovation in the economy than just new 
products or processes. In recent years, an 
international policy discussion has centred 
on the idea of national and regional 
innovation systems as the key to 
understanding and improving innovation 
performance. The Organization for 
Economic Co-operation and Development 
(OECD) has played a major role in 
promoting this view, which stresses that 
flows of technology and information 
among people and institutions are key to 
the innovative process in national and 
regional economies. Innovation results 
from a complex set of relationships among 
participants in the system, which is made 
up of businesses, universities and 
government research institutes (see 
Exhibit 19.2). 


19.12 As aresult, the rate of 
technological change in a country and the 
international competitiveness of 
companies do not depend simply on the 
scale of research and development. They 
also depend on how available resources 
are managed and organized, at both the 
enterprise and the national level; in other 
words, they depend on the structure that 
encourages and exploits innovations. 


19.13 For policy-makers, an 
understanding of national and regional 
innovation systems can help identify 
leverage points for enhancing innovative 
performance. The recommended policies 
tend to be those that improve the system 
itself by building networks of institutions 
and that aim at improving the innovative 
capacity of firms, particularly their ability 
to identify and absorb technologies. An 
important result of this policy discussion 
is that innovation performance 1s now 
regarded as more than simply the result of 
research and development spending. How 


A good understanding 
of innovation systems 
is important in making 
policy decisions. 
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Exhibit 19.1 


Models of Innovation 


In recent years, the traditional linear model of innovation has been superseded by new analytical frameworks. 


The Linear Model of Innovation 


Basic _ Applied __, =~ Technology Product/Process : Production Marketing 


Research Research Development Development 


Source: Industry Canada, 1996-97 Performance Report 


The Chain-Link Model of Innovation 


Research 


eS / x y 
/ x Knowledge ve ; 
\ 
| / y ro a! 
Potential ; Invent Detailed Redesign Distribute 
Market and/or Design and | and Produce | and Market 
Produce : Test ; 
| Analytical ; 
| Design ; 


Source: Kline S.J. and N. Rosenberg (1986), “An Overview of Innovation”, in R. Landau and 
N. Rosenberg (eds). The Positive Sum Strategy. Harnessing Technology for Economic Growth, 
National Academic Press, Washington, DC, p. 289. 


An Open System Model of Innovation 


Knowledge/ 
Available 
Technologies 


Research 


——— 
<___ > 


A) Product Development << Design 
== Marketing Production Marketing 


B) Technology Acquisition ~<———*» Management 
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Source: Government of Quebec, Conseil de la science et de la technologie 
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well technology is diffused and adopted 
throughout the economy is as important 
as, if not more important than, its creation 
in any one particular “high-tech” sector. 


19.14 Innovation as public policy. As 
a matter of public policy, Canada’s 
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innovation performance has aroused 
concern for many reasons. First, many 
observers argue that Canada’s relatively 
weak economic growth over the last two 
decades is attributable to a marked decline 
in the rate of productivity increase. It is 
widely held that innovation and its 


Framework Conditions Exhibit 19.2 


— Basic education system 
— Communications infrastructure 


— Financial institutions 


— Legislative and macro-economic settings National Systems of Innovation 


— Market accessibility 


— Industry structure 


— Linkages between firms 


— International links 


Transfer Factors 


— Presence of technology gatekeepers 


— Mobility of experts 
— Codified knowledge 


— Ethics, values (trust, openness) 


—— 


_- 


yw 


/ — University system 


— Non-appropriable innovation support (funding and 
institutions directed toward areas with particularly high _/ 


externalities risks) 


Innovation Dynamo 


R&D Non-R&D 
— Basic research — Identification of new product concepts and production 
~ Strategic technologies 
research — Development of pilot or full-scale production facilities 
— Development — Purchase of technical information (patents, know-how, 


consulting services) 
— Human skills development 


— Investment in process equipment and intermediate 
inputs embodying innovation of others 


— Changes in the management systems and the overall 
production system 


5 . . aN 
a Science and Engineering Base oa 


— Specialized technical training system Pte 


— Support system for basic research \ 
— Public good R&D activities (funding and programs directed | 
toward areas such as health, environment or defence) | 

| 

— Strategic R&D activities (funding programs and institutions / 
directed toward “‘pre-competitive R&D”) 


Source: Conference Board of 
Canada, Performance and 


Potential 1997 
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Studies of relative 
economic performance 
have identified an 
“innovation gap” in 
Canada. 


diffusion in the economy plays a role in 
determining this rate. For example, in 
summarizing the factors underlying the 
relatively poor productivity performance 
in Canada, a 1996 report by the 
government’s policy research initiative 
concludes: “In simple terms...[it] could be 
linked to a lack of adjustment and 
innovation.” In a similar vein, the 
Conference Board of Canada’s 
Performance and Potential (1997) 
maintains that technology diffusion has a 
decisive impact on the growth of 
productivity. 


19.15 A second concern about 
innovation as public policy relates to the 
changing global economy. Many observers 
contend that the global economy is 
undergoing a fundamental restructuring 
that is ushering in a new economic 
paradigm known as the knowledge-based 
economy. According to this view, to 
survive and prosper in the 
knowledge-based economy, firms, 
industries and economies must adopt a 
strategy of continuous innovation and 
must engage in innovation competition. 
The application of new knowledge and 
ideas and the creation of new products and 
processes is the essence of the 
knowledge-based economy. 


19.16 Concern about Canada’s capacity 
for innovation leads to a number of public 
policy questions. Are we innovating 
widely enough and fast enough? Are we 
doing as well as other countries? The last 
question is frequently framed in terms of 
an “innovation gap” between Canada and 
its major economic rivals. 


19.17. The innovation gap. The idea of 
a gap is a popular one in the context of 
studies of relative economic performance. 
For example, numerous empirical studies 
have attempted to account for the 
productivity gap between Canada and the 
United States. Recently the notion of an 
innovation gap, coined by the OECD, has 
gained acceptance. 


19-10 


19.18 The term is commonly attributed 
to the OECD’s 1995 Economic Survey of 
Canada that stated: “The view is widely 
held that Canadian industry suffers from 
an innovation gap compared with other 
developed countries.” The OECD 
attributed the gap to insufficient efforts in 
research and development, as well as to 
private and public sector attitudes that 
have contributed to an inflexible and 
non-innovative industrial structure. 
Exhibit 19.3 provides information on 
different aspects of Canada’s innovation 
performance. 


19.19 Closing the gap. In the fall of 
1994, the government set out its economic 
program in Agenda: Jobs and Growth. 
One of the four papers released, Building 
a More Innovative Economy, focussed on 
four policy areas — marketplace climate, 
trade, infrastructure and technology. The 
1996 Budget built upon the priorities of 
the jobs and growth agenda and, in 
particular, announced the creation of 
Technology Partnerships Canada as part of 
the government’s intention to make 
Canada one of the most innovative 
countries. 


19.20 = Also in 1996, the federal 
government’s strategy Science and 
Technology for the New Century asserted: 
“itis well documented that, by 
international standards, Canada’s S&T 
effort has considerable room for 
improvement, particularly in the 
development, adoption and 
commercialization of technology.” It goes 
on to link the innovation gap to poor 
productivity performance, and the poor 
productivity performance to depressed 
real incomes and high government 
deficits. 


19.21 Industry Canada’s 1998-99 
Report to Parliament on Plans and 
Priorities points to the innovation gap and 
states that failure to capitalize on 
innovation has been a major reason for 
Canada’s relatively slow productivity 
growth over the past two decades. The 
Department’s 1997—98 Performance 
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Report warns that without innovation, real 
income growth will not meet Canadians’ 
expectations. It goes on to state: “While a 
sizable overall innovation gap remains, it 
has been reduced over time. The 
government has supported innovation 
through a number of key initiatives, such 
as the Canada Foundation for Innovation, 
Technology Partnerships Canada, 
Networks of Centres of Excellence and 
the Industrial Research Assistance 
Program.” However, as noted in 

exhibits 19.1, 19.2 and 19.3, innovation is 
a multifaceted idea. Most of the existing 
information is on inputs to innovation, but 
not on innovation performance directly. 


19.22. Exhibit 19.4 explains some of the 
challenges involved in measuring 
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innovation. What this information means 
in aggregate is not at all clear. For 
example, the 1996 report by the 
government’s policy research initiative 
argues that our understanding of what is 
happening in the knowledge-based 
economy is constrained by the extent and 
quality of current indicators. The report 
states that new indicators are needed that 
capture the innovation process and the 
distribution of knowledge among key 
organizations. 


19.23. Toward that end, Statistics 
Canada is currently developing indicators 
of science and technology activity and a 
framework to tie them together into a 
coherent picture. Several key areas are 
covered by this initiative, such as 


This exhibit stresses the idea that innovation performance has many facets that can be measured 
with varying degrees of difficulty. Some of the available information relating to a variety of 
apparent facets of innovation performance has been assembled as follows: 


e Canada’s research and development expenditures as a proportion of gross domestic product 


(GDP) are among the lowest of the Organization for Economic Co-operation and Development 
countries, both in the private sector and overall. Not only is private sector spending on research 


share of real GDP. 


and development in Canada relatively low but a handful of firms do a disproportionate share. 


e Canada lags behind the U.S. by 35 percent in investments in machinery and equipment as a 


__e When complementary innovations such as those in organizational structure and in human 
resource management practices are considered, the picture is somewhat worse. A 1994 study, 
based on firms adopting computer-based technologies and three kinds of organizational 
innovations, showed that while the vast majority had adopted the hard technologies, a minority 
had adopted complementary innovations such as employee participation programs, job design 


programs and pay-for-performance programs. 


e A recent Statistics Canada paper asked, “Do small firms suffer from an innovation gap?” Small 
firms were only half as likely as larger ones to have introduced product or process innovation. 
Furthermore, smaller firms were much less likely to introduce process innovations. 


e Canadian patenting activity lags behind that of the U.S. by as much as a half. 


e Canada leads the U.S. in the proportion of the work force with some post-secondary training, 


but lags in the proportion of university degrees. 


Recent information suggests that some gaps appear to be getting smaller. For example, Canada is 
now tied with the United States in the proportion of graduates receiving degrees in natural 
sciences, mathematics, computer science, engineering and architecture. However, the rate of 
change is slow and it is not clear why these gaps are narrowing. 


To sum up, depending on the measures used, various gaps emerge between Canada and one or 
more trading partners. What the relative significance of each of these gaps is or what they mean in 
aggregate has yet to be clarified. Careful interpretation of available information is essential. 
Traditional indicators of inputs (such as R&D expenditures, number of research workers) and 
outputs (such as patents) fail to measure fully countries’ innovativeness and trends in innovation 


performance. 


Most of the existing 
information is 

on inputs to 
innovation rather 
than on innovation 
performance directly. 


Exhibit 19.3 


Evidence of Canada’s 
Innovation Gap 
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innovation, technology diffusion, human 
resources in science and technology and 
interrelations among different institutions. 


19.24 Innovation and productivity 
growth. Innovation and productivity 
growth go hand in hand. Labour 
productivity is a measure of the quantity 
of goods and services produced per unit of 
labour input. When output per unit of 
input rises, it is possible for some people 
to consume more without others 
consuming less, or even for everyone to 
consume more. In other words, higher 
productivity means a higher standard of 
living. Innovation, in turn, entails 
changing the goods and services we 
produce and how we produce them. 
However, the rate of labour productivity 


countries has declined dramatically since 
1973. Thus, while this phenomenon is not 
unique to Canada, a number of recent 
reports have concluded that a productivity 
gap exists between Canada and the United 
States. 


19.25 Recent studies identify a number 
of factors accounting for our relatively 
poor productivity performance in relation 
to the United States: slower and weaker 
adjustment to the two energy price shocks, 
a slower rate of capital accumulation, a 
slower rate of adoption of best practice 
technologies, a slower rate of growth in 
research and development spending, 
weaker adjustment to the 
knowledge-based economy, and weaker 
competition in both product and factor 


growth in Canada and other industrialized markets. Some commentators have also 


Exhibit 19.4 


Can Innovation Be Measured? 


There is a wide range of indicators of innovation or “innovativeness” in an economy and, depending on which one is used, a 
different sort of innovation gap emerges. 


One of the most frequently cited measures (and the one that figured prominently in the gap analysis of the Organization for 
Economic Co-operation and Development) is research and development (R&D) expenditures, expressed as a percentage of the gross 
domestic product. Numerous variants of this measure distinguish between public and private expenditures, civil and defence, 
performers and funders, and so on. But the fundamental problem with the various measures is that R&D may be a necessary 
condition for improving innovation performance, but it is not a sufficient one. Research and development is an input into the 
innovation process, not an output. However, one of the most common output indicators — number of patents — is difficult to 
interpret since the propensity to patent varies considerably across countries and even across industries within the same country. 


Alternatively, speed, flexibility and timing are hallmarks of innovation. So the speed with which firms acquire and apply new 
technologies, the “speed to market” new products, and the percentage of sales accounted for by new products are also relevant 
indicators. 


Next, many firms that do not undertake R&D nevertheless use technologically sophisticated inputs in their production process. One 
approach to the measurement of technological innovativeness is therefore to first develop a ranking of the technological 
sophistication of intermediate inputs and then, using an input-output framework, characterize industries as to their degree of 


technology use. A somewhat analogous approach can be used to classify industries into high-, medium- and low-knowledge 
intensities. 


The word “knowledge” in the term “knowledge-based economy” suggests a variety of measures addressed to the people side of the 
innovation equation. These might include proxies for skill levels such as the educational attainment of the work force, the proportion 
of “knowledge workers”, research and development researchers per 10,000 in the labour force, etc. Expenditures on and enrolments 

| in universities are a crude but comparable indicator of the knowledge infrastructure. 


| Its also important to distinguish innovation reflected in the use of hard technologies (robots, computers, lasers, etc.) from soft or 
complementary technologies (organizational change). 


In summary, innovation can probably be measured but no one measure will do. A balanced assessment of innovation performance 
requires reference to a comprehensive range of indicators. 


19-12 Report of the Auditor General of Canada — September 1999 


cited relative tax rates as a key reason 
why American productivity growth has 
outstripped our own. Others cite smaller 
scales of operation, and a low Canadian 
dollar shielding less productive firms. 
Although spending on research and 
development has some role to play, there 
is clearly no one factor that explains this 
productivity gap. 


19.26 Summing up. It is difficult to do 
justice to these broad economic issues 
within the context of this chapter; 
however, a number of observations can be 
offered: 


e Assessing the innovation 
performance of the economy is a 
multifaceted challenge for which little 
comprehensive information yet exists. 


e [tis reasonably clear that better 
innovation performance depends on more 
than just spending on research and 
development; it involves supporting 
research and development with the 
activities needed to embed new 
technologies in the economy. 


e Spending on research and 
development is not the only determinant 
of the Canadian economy’s rate of 
productivity growth, and may not be the 
most important one. 


Focus of the audit 


19.27. We audited the following 
programs: 


e Industrial Research Assistance 
Program — National Research Council 


e Research Partnerships Program — 
Natural Sciences and Engineering 
Research Council 


¢ Networks of Centres of Excellence 
— NSERC 


e« Technology Partnerships Canada — 
Industry Canada 


These programs represent most of the 
funding within the Industry Portfolio 


Industry Portfolio — Investing in Innovation 


directed to improving the innovation 
capacity of industry. 


19.28 With the previously noted issues 
in mind, we examined whether 


anag : i i 
managemen Better innovation 


e could show that these programs are 
designed to improve Canada’s innovation 
performance; on more than just 
research and 
development 


spending. 


e exercises due diligence in investing 
in innovation projects; and 


e knows if the programs are achieving 
value for money. 


19.29 This audit brings attention to 
what constitutes due diligence in assessing 
applications for grants and contributions. 
In our view, the assessments supporting 
decisions to make grants and contributions 
need to be as thorough as the 
circumstances require and they need to be 
documented so that subsequent review and 
performance measurement are possible. 
Due diligence does not imply exhaustive 
analyses in all cases; it simply means 
ensuring that funding decisions take all of 
the project assessment criteria into 
account and that they are based on reliable 
information. Further details on the audit 
can be found in the section About the 
Audit at the end of the chapter. 


Observations and 
Recommendations 


Addressing the Innovation Gap 


19.30 In 1996 the government 
introduced its strategy, Science and 
Technology for the New Century. One of 
the purposes of the strategy was to 
improve the governance and management 
systems for federal science and 
technology (S&T) so as to make the 
government a more effective partner in the 
country’s innovation system. The Minister 
of Industry, in response to the S&T 
strategy, set out the following role for his 
portfolio: “The Portfolio will use its 
unique tools and capabilities, maximizing 


performance depends 
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There is no strategy or 
framework for any of 
the programs we 
audited that links 
spending decisions 

to an innovation gap 
or gaps. 
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linkages and partnerships, to help 
Canada become a world leader in 
knowledge-based innovation that will 
result in jobs, exports and economic 
growth.” 


19.31 This message is repeated in 
various ways 1n many recent reports to 
Parliament by Industry Canada, the 
Natural Sciences and Engineering 
Research Council, and the National 
Research Council. For example, the 
government’s 1998 report on federal 
science and technology, Building 
Momentum, claims that Technology 
Partnerships Canada has become an 
effective tool for closing the innovation 
and productivity gaps; the 1999-2000 
Report to Parliament on Plans and 
Priorities of the Natural Sciences and 
Engineering Research Council states that 
its financial support for university- 
industry research projects leads to new 
products, processes and services; and the 
National Research Council (NRC) reports 
that the Industrial Research Assistance 
Program (IRAP) activities stimulate the 
innovative capacities of Canadian small- 
and medium-sized enterprises. 


19.32 Accordingly, we looked at 
whether management had a strategy that 
linked the funding of individual projects 
carried out by or in partnership with 
business to specific aspects of the broader 
government goal of improving innovation 
performance in the economy. In particular, 
we wanted to know how these programs 
were supposed to address specific gaps in 
innovation performance and to what 
extent. 


19.33 In supporting innovation, a 
number of overlapping roles are possible, 
including: 


e encouraging the invention and 
introduction of new or improved 
technology; 


¢ making firms aware of existing 
technologies; 


e adapting existing technology to new 
uses; and 


e helping firms improve their 
innovation capacity — that is, helping 
them improve the assets, processes and 
tools that keep them competitive. 


19.34 In addition, programs may 
support business innovation by 
contributing to national and regional 
innovation systems — for example, by 
supporting the training of university 
students through funding for 
business-university projects. 


19.35 Each program supports 
innovation. Each of the programs we 
audited makes grants or contributions 
(IRAP provides advice as well) to carry 
out one or more of these roles. We found 
that management of each program is 
developing goals and measures for 
assessing performance in relation to these 
roles. However, these goals and measures 
are not yet used to manage program 
activities. In the case of the Networks of 
Centres of Excellence program, efforts 
have been made to establish broad goals 
that capture the program’s intended 
impact on innovation performance. 
Nevertheless, the expected results for 
these programs are not expressed 1n 
innovation performance terms — in 
particular, in terms that capture the 
intended impact on innoyation systems. 
Clear statements of expected results are 
prerequisites to sound program design and 
management, and to measuring and 
interpreting the actual results achieved. 


19.36 While each program undoubtedly 
contributes to improving innovation 
performance, there is little information on 
the extent of that contribution or its 
significance. We found that no strategy or 
management framework (with goals and 
targets) existed for any of these grant and 
contribution programs, linking spending 
decisions to an innovation gap or gaps. 
Nevertheless, program management has 
taken steps to develop broad approaches 
to supporting innovation. Technology 
Partnerships Canada, for example, has 
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developed a plan explaining what kind of 
projects it wants to support whereas [RAP 
takes a decentralized, community-based 
approach that is intended to be reactive 
and industry-driven. However, these initial 
steps are general in nature, and are not yet 
associated with clear results and 
performance measures. Before adequate 
management frameworks can be 
developed, certain basic questions have to 
be answered. What improvement in 
innovation performance, and in which 
aspects, is expected from the money spent 
either on a program or on a 
project-by-project basis? How has 
innovation performance actually improved 
as a result of program spending? Until 
management is able to answer these 
questions, it cannot be confident that it is 
achieving the most value for money in 
funding particular projects. 


19.37. Toward a coherent portfolio 
approach. The government’s 1996 S&T 
strategy, and subsequent reports to 
Parliament, imply a focussed, 
co-ordinated approach by the Industry 
Portfolio to addressing innovation 
performance problems in the Canadian 
economy and thereby improving the rate 
of productivity growth. While the 
Portfolio is working on such an approach, 
we believe that it needs further 
development. Moreover, management 
recognizes that there is an important 
opportunity for Industry Portfolio 
programs to work in concert. To do this, 
management needs to agree on what 
innovation performance issues can be best 
addressed by the Portfolio and what 
results are expected from each of its grant 
or contribution programs. 


19.38 Good program design is at least 
partly based on good information and on 
an analysis of the issues being addressed 
— why they exist, how important they 
are, and what can be done. As discussed in 
the introduction to this chapter, there are a 
number of fundamental issues concerning 
innovation performance for which good 


Report of the Auditor General of Canada — September 1999 


Industry Portfolio — Investing in Innovation 


information does not yet exist, although 
Statistics Canada is working on gathering 
this information. By working 
collaboratively, management in the 
Industry Portfolio will be able to improve 
the available information as a basis for 
good program design. 


Management needs 

to agree on 

what innovation 
performance issues 
can be best addressed 
by the Industry 
Portfolio. 


19.39 Industry Canada, the National 
Research Council and the Natural 
Sciences and Engineering Research 
Council should provide Parliament with 
information on the following issues: 


e What does innovation performance 
of the economy mean? 


e What specific innovation 
performance problems are Industry 
Portfolio programs supposed to 
address? 


e What specific results are expected 
from Industry Portfolio programs 
toward addressing these problems? 


e Are these expected results being 
achieved? 


Industry Canada, National Research 
Council and Natural Sciences and 
Engineering Research Council joint 
response: The Industry Portfolio is 
engaged in a long-term effort to focus its 
reporting to Parliament on performance 
measures. This effort will become evident 
as the Reports on Plans and Priorities and 
the Performance Reports of Industry 
Portfolio departments and agencies evolve 
in the future. It is important to recognize, 
however, that few programs in 
government, and even fewer departments 
and agencies, exist with a single objective. 
For many of the Industry Portfolio 
programs that address one or more facets 
of Canada’s innovation gap, other, 
complementary objectives are also 
important. The various dimensions of the 
innovation gap between Canada and our 
competitor nations are being addressed to 
bring about the long-term result that 
knowledge is put to new use in more 
organizations in the private, public and 
not-for-profit sectors more frequently, and 
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more quickly. The strategy to bring this 
about includes: 


e documenting the role innovation 
plays in Canadians’ standard of living and 
quality of life and promoting its 
understanding, throughout Canadian 
society; 


e increasing the creation of knowledge 
in Canada and improving access to and 
diffusion of knowledge created in Canada 
and abroad; 


e encouraging the early consideration 
of applications of knowledge and speeding 
its application to processes, products and 
services; and 


e ensuring that Canada’s market 


frameworks encourage the development of 


knowledge and innovation. 


The success of this strategy depends on 
several factors. One key factor is the 
development of highly qualified people 
with the skills and education to create new 
knowledge and transfer that knowledge to 
those who will put it to use. A second 
success factor is the strengthening of 
national and regional innovation systems. 


Statistics Canada is involved in 
developing a set of indicators of 
innovation performance, and private 
sector organizations, for example, the 
Conference Board of Canada, are 
developing complementary approaches to 
monitoring Canada’s innovation 
performance. These are being studied and 
adapted in designing and refining 
appropriate methodologies for 
performance measurement in the Industry 
Portfolio. 


Industrial Research Assistance 
Program 


19.40 The National Research Council 
delivers the Industrial Research Assistance 
Program (IRAP), which helps small- and 
medium-sized enterprises (SMEs) develop 
and exploit technologies. In 1998-99, 
IRAP had a budget of $120 million, 


$76 million of which was spent in 
contributions to individual SMEs. 


19.41. IRAP has a field staff of about 
260 industrial technology advisors, 
located in 90 communities across Canada, 
who assist SMEs in resolving their 
technology-related problems, or in 
exploiting opportunities. About 30 percent 
of the technology advisors are National 
Research Council employees. The rest 
work in over 130 public and private sector 
organizations that participate as IRAP 
network members. IRAP has contribution 
agreements with each network member for 
the salaries and other expenses of network 
technology advisors. 


19.42 [RAP also co-ordinates the 
Canadian Technology Network, which 
comprises 1,000 members nationally and 
is an informal affiliation of organizations 
that provide assistance to SMEs. We did 
not examine IRAP’s role in the Network. 


19.43 We selected a sample of 120 
contributions to firms from 1994 to 

June 1999. The amount of funding 
provided ranged from $1,000 to $998,000. 
We also examined selected contribution 
agreements with network members. 


19.44 In 1996, IRAP went through an 
extensive strategic planning exercise. At 
the same time, a separate review of the 
program concluded that although IRAP 
was successful in helping SMEs become 
more innovative, management practices 
needed significant improvement. The 
objectives and direction set out in IRAP’s 
new strategic plan led to specific changes 
to the terms and conditions of the program 
in 1998 for its contributions to firms and 
to network members. Although IRAP had 
focussed for many years on building the 
innovation capability of SMEs through 
improving their technological 
competence, this role was made explicit in 
its new objective and in the project 
assessment criteria (see Exhibit 19.5). As 
a result of these recent changes, IRAP was 
in a period of transition during our audit 
as it developed and applied the tools its 
technology advisors need to assess and 
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help firms make improvements in 
innovation capacity. 


Concerns about support for funding 
decisions 


19.45 The current objective of IRAP’s 
contributions to firms is to stimulate 
innovation in Canadian small- and 
medium-sized enterprises and to enhance 
their innovation capabilities by enabling 
recipients to undertake innovation-related 
activities they would not otherwise be able 
to undertake. 


19.46 We expected that IRAP’s funding 
decisions would take into account all the 
criteria established for assessing potential 
projects and that these decisions would be 
based on reliable information. We 
recognize that many contributions under 
IRAP are for relatively small amounts, 
and an exhaustive analysis should not be 
required. For example, 43 of the projects 
we audited received less than $15,000 in 
funding. Nevertheless, we expected that to 
meet the requirements of due diligence, 
there would be at least minimal 
information supporting the assessment of 
each of the criteria for all contributions. 


19.47 We found that only about 

15 percent of the funding decisions were 
based on an assessment of all of the 
project criteria, mainly because the need 
for IRAP support had not been considered. 
We also found a range of practices in 
assessing the other criteria — 1n 
particular, how projects would improve 
the technological competence of funding 
recipients. In our view, the assessment of 
individual funding criteria for many 
projects was not sufficiently thorough. 


19.48 Little support for need for 
funding. We expected that the project 
files would provide an explanation of why 
IRAP funding was necessary for the 
project to proceed, since this is a key test 
of the need for government financial 
support. This test is commonly referred to 
as incrementality, because it means 
funding worthwhile projects that would 
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not have proceeded otherwise with similar 
results. However, we found that the need 
for IRAP funding was not explained for 
84 percent of the projects. In these cases, 
we could find no evidence that 
incrementality was considered in IRAP’s 
funding decision. We found examples 
where the companies receiving assistance 
have annual sales ranging from $6 million 
to $31 million and yet have received 
funding ranging from $1,000 to $225,000. 
Although these firms might have been 
new to research and development, it was 
not explained why they needed IRAP 
assistance for their projects. 


19.49 Assessments often limited to 
technical feasibility. As discussed 
previously, IRAP changed the assessment 
criteria in 1998 to explicitly include 
improving the innovation capability of 
SMEs. Nevertheless, for some time before 
that, IRAP management had formally 
stated that it wanted to support projects 
that had an enduring impact on the 
technical competence of the firm rather 
than a one-time process or product 
innovation. Under the previous assessment 
criteria, for example, the assessments of 
applications for funding were supposed to 
judge how much improvement a project 
would make in the technology base of the 
firm. 


For over 80 percent of 
the files we audited, 
the need for 
government funding 
was not explained. 


Exhibit 19.5 


Industrial Research Assistance Program — Assessment Criteria 


All financial assistance to business under the Industrial Research Assistance 
Program must be assessed against these basic criteria: 


e the willingness and ability of the recipient to enhance the innovation capability 
of the recipient or the firms it represents: 

e the potential impact of the proposed project on the recipient’s innovation 
capability; 

e the potential impact of the project on the recipient’s competitiveness or on that 
of the firms it represents; 


e the potential socio-economic benefit of the project for Canadians; 
e the need for financial contribution; and 


e the degree and nature of the uncertainty and risk. 


Report of the Auditor General of Canada — September 1999 


19-17 


a a RR A SR A SS AT 


Industry Portfolio — Investing in Innovation 


There was a wide 
range of practices in 
assessing benefits to 
the firms. 
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19.50 For most of the projects we 
audited, the assessments of technical 
feasibility were reasonably complete and 
thorough. What was less clear was how 
much projects would increase the 
technological competence or innovation 
capability of the funding recipients. We 
found a wide range of practices in how 
technological competence or innovation 
capability were defined and considered in 
project assessments. We also found that 
technology advisors have not yet been 
provided with the tools they need to assess 
and help with improving the innovation 
capacity of client firms. 


19.51 Expected business benefits not 
always considered. In the project 
assessments, the expected benefits from 
the proposed projects are expressed as 
increased sales, cost reductions or jobs. 
These projections are used as indications 
of increased competitiveness, which is one 
of the funding criteria for the program. We 
appreciate that detailed projections of 
sales or cost reductions are too speculative 
to provide reliable support to research and 
development (R&D) proposals. However, 
we expected that, at the very least, the 
plausibility of the projected sales and 
market potential for the end products 
would be challenged. Instead, we found a 
range of practices in assessing benefits to 
firms. For about 10 percent of the 
projects, we could not satisfy ourselves 
that these benefits had been properly 
considered in the funding decisions. 


19.52 As mentioned previously, the 
depth of analysis underlying a funding 
recommendation varies according to the 
significance of IRAP’s financial 
involvement. For larger projects, expected 
benefits are normally better explained and 
substantiated. However, regardless of the 
amount of funding provided, we believe 
that IRAP has not properly assessed the 
merit of a project unless it examines the 
rationale for expected business 
performance improvements. 


19.53. IRAP asks applicants to identify 
socio-economic benefits to Canada from 
their projects. Applicants normally 
identify desirable business impacts, such 
as increases in their firms’ revenues and 
work force or at least the maintenance of 
existing jobs. Jobs created or maintained 
are considered a socio-economic benefit. 
We found that expectations of job creation 
or maintenance were not adequately 
supported, even taking into account the 
uncertainties in R&D outcomes. 


Mixed program performance 
information 


19.54 Management has made various 
efforts to assess IRAP’s performance over 
the past several years. Despite the 
limitations and ambiguities 1n 
performance information, it appears that 
IRAP has contributed to the technological 
development of many SMEs. However, 
when we consider the available 
performance information along with the 
weaknesses in project assessment that we 
found, it is clear that IRAP’s performance 
could be improved. 


19.55 Incomplete information on 
project results. We expected that IRAP 
would know if funded projects were 
achieving the expected technical and 
commercial results. However, there was 
often little information on file concerning 
the success of projects. Although final 
reports on the technical performance of 
projects are required, about one third of 
the completed projects we audited did not 
have one. 


19.56 In addition, we noted that some 
regional offices require technology 
advisors to report on results an average of 
six months after project completion. These 
reports were available for 60 of the 90 
completed projects. While these reports 
provide some information on project 
results, such as the number of jobs 
created, there is no explanation of what 
produced these results or how the 
innovation capacity of the firms was 
improved. Nevertheless, we also found 
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other information on file suggesting that 
the majority of projects had at least met 
their technical goals of developing 
technology. 


19.57 Results of advisory service 
unclear. As discussed previously, IRAP 
has about 260 technology advisors, 

70 percent of whom are not National 
Research Council employees. Although 
technology advisors spend a considerable 
portion of their time providing advice to 
clients, IRAP has not set expected results 
for its advisory services, nor does it track 
the results. We expected that IRAP would 
at least be aware of the extent to which 
the information needs of its clients were 
being properly identified and met. 


19.58 Under the agreements signed 
with member organizations, network 
technology advisors are expected to set 
annual objectives and report performance 
toward those objectives monthly and 
annually. However, we found that 
expected and actual performance are 
described almost entirely as activities 


(number of clients visited, telephone calls, 


networking activity, professional 
development) and not as results. In other 
words, it is unclear what results the 
advisory services expect to or actually do 
achieve. IRAP management is not certain 
what help was requested by its clients and 
what its advisors did for them. 


19.59 Incrementality needs 
clarification. IRAP regularly surveys the 
companies it funds and asks what 
difference its contributions and advice 
made. In our view, the results of these 
surveys suggest that the majority of the 
projects funded by [RAP may have gone 
ahead anyway without IRAP funding or 
advice. It appears that IRAP’s financial 
support facilitates projects rather than 
making the difference between whether 
they proceed or not. 


19.60 | We examined the results of these 
client surveys from 1996 to 1998. In each 
of these years, about 40 percent of 
respondents indicated that their project 
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had needed IRAP funding to proceed. The 
remaining 60 percent reported that they 
would have proceeded anyway without 
IRAP funding, although perhaps more 
slowly or with added difficulty. A separate 
survey in 1996 found that the majority of 
respondents considered IRAP support 
helpful for their projects. However, 

40 percent indicated that they would have 
gone ahead if IRAP funding had been 
smaller; and 17 percent did not know. 


19.61 = What is unclear from the results 
of these surveys is whether the projects 
would have proceeded with the same 
scope and achieved similar results. 
Undoubtedly, IRAP funding mitigates 
some of the risk for these firms by 
reducing their direct investment and 
therefore potential losses if the project 
fails. However, mitigating the risk of 
projects that would have proceeded 
anyway 1s not a formal objective of the 
program. 


19.62 In addition, IRAP asked firms 
that had received funding for their projects 
about the usefulness of its advisory 
services. Only 13 percent of survey 
respondents reported that the project 
would not have proceeded if technical 
assistance had not been provided; 

30 percent of recipients indicated that 
IRAP advisory services had no effect and 
the remainder reported that they would 
have gone ahead with the project, 
although with added difficulty. 


19.63 This survey information on 
performance, while insufficient to support 
a definitive conclusion, suggests that 
many projects might not have needed 
IRAP’s funding or advice to proceed. The 
true incrementality of IRAP support needs 
further investigation. 


19.64 Job creation may be overstated. 
IRAP contributes to job creation in two 
ways. First, direct jobs may be created 
when technical staff are hired by the client 
to work on an IRAP-supported project. 
Second, new jobs may be created from the 
commercialization of a product or process 


Client surveys suggest 
that 60 percent of the 
projects would have 
proceeded without 
funding under the 
Industrial Research 
Assistance Program. 
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Benefits of the 
projects, such as jobs 
created, may have 
been overstated. 
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resulting from a project to which IRAP 
was one of the contributors. 


19.65 In recent public reports, IRAP 
has claimed that it creates between 9,000 
and 10,000 jobs a year. This figure is 
based on a 1995 study that IRAP did on 
jobs created, saved or lost through all 
projects it had supported that terminated 
in fiscal year 1991-92. Many were 
multi-year projects, and this was taken 
into account in the study. 


19.66 Our review of the methodology 
followed in this study suggests that the job 
creation numbers may be unreliable for 
the following reasons. First, the 
elimination or loss of jobs in other firms is 
not considered. Second, incrementality 
(whether IRAP funding was needed for 
the project to proceed) and attribution (the 
extent to which the jobs can be attributed 
to IRAP’s support for a project) may not 
have been adequately considered, leading 
to an overestimation of IRAP’s role in job 
creation. In our view, any job creation and 
maintenance numbers reported by IRAP 
need careful explanation and qualification. 


19.67 Limited program evaluation. 
We examined two evaluation reports, 
respectively completed in 1990 and 1996, 
to determine whether IRAP had 
adequately assessed its performance. Both 
reports concluded that there is clear 
evidence that IRAP is successful in 
helping SMEs become technically 
capable. Management has informed us 
that IRAP’s role in supporting Canada’s 
innovation system was left aside in the 
1996 assessment to concentrate on other 
issues. In our view, these evaluations 
provide only a limited perspective on the 
relevance, success and cost-effectiveness 
of IRAP. 


19.68 
should: 


The National Research Council 


e establish clear expected results for 
the Industrial Research Assistance 
Program (IRAP) and report 
performance against them; 


e ensure that the decision to fund a 
project is based on an appropriate 
assessment of the project’s merits and 
the firm’s need for IRAP support; 


e ensure that reliable information is 
gathered on the results of individual 
projects; 


e establish results-based plans and 
performance measures for its advisory 
services; and 


e conduct a program evaluation of 
IRAP that addresses all of the key 
evaluative questions. 


National Research Council’s response: 
We note with satisfaction the chapter's 
expressed view that “IRAP has 
contributed to the technological 
development of many SMEs.” This view is 
very strongly held by IRAP clients and by 
many other observers. Across all regions 
of Canada, IRAP’s network of Industrial 
Technology Advisors provides extensive 
technical and business advisory services 
and, where necessary, financial assistance 
in nurturing companies and in assisting 
them to develop R&D projects. 


We agree with the recommendations to 
improve the measurement of IRAP ’s 
performance, and share the opinion that 
we should provide industrial technology 
advisors and managers with the tools and 
information needed to make quality 
decisions, report on performance and 
manage the program effectively. We are 
already acting on the recommendations, 
and indeed some of the changes were in 
the process of development at the time of 
the audit, as outlined below: 


e A program performance framework 
was approved in 199S—99. Under active 
development are: a model to understand 
IRAP’s role in the innovation process, data 
collection instruments, assessment of 
incrementality, and plans to capture 
performance information, including a 
follow-up system for IRAP projects to 
record the downstream benefits against 
IRAP objectives. 
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e An upgraded quality assurance 
process is being implemented across the 
country and is providing information on 
due diligence in the project 
decision-making process, including the 
application of selection criteria. One 
result, initially for funded projects, will be 
a more visible record of the need for and 
anticipated benefits of IRAP. assistance. 


e« A major business process 
improvement effort was initiated in 
April 1998 to modernize the IRAP client 
business process. This will enable the best 
use of information to support decision 
making close to clients, enhance 
collaboration, encourage sharing and 
learning, as well as measure impact and 
results. This process will continue to be 
the focus of our efforts. 


e An information management plan 
and strategy is being prepared. This will 
guide the refinement of existing and 
development of new IRAP systems to be 
compatible with the recent adoption by 
NRC of a new large-scale, integrated 
enterprise business information system. 
The plan and strategy cover a 
comprehensive electronic database for 
project approval processes, project 
management, network interactions and 
project results. 


e As part of its normal assessment 
schedule, NRC is planning to undertake 
an evaluation of IRAP, which will include 
addressing questions in the Auditor 
General’s Report. 


The IRAP program is in the midst of 
significant change as we Strive to put in 
place new services, processes and systems 
to enhance the innovation capabilities of 
SMEs while building on our primary 
approach, which, as noted in the chapter 
“is decentralized...community-based... 
reactive and industry-driven.” We are 
committed to continuing progress and 
improvement. 
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Research Partnerships Program 


19.69 The Natural Sciences and 
Engineering Research Council (NSERC) 
was created in 1978 and is a major source 
of funding for university-based science 
and engineering research and training in 
Canada. It is organized around two 
program directorates: Research Grants and 
Scholarships, and Research Partnerships. 


19.70 Our audit focussed on the 
Research Partnerships Program, which 
accounted for $95 million of NSERC’s 
expenditures in 1998-99, and the 
Networks of Centres of Excellence 
program, which had expenditures of 
$47 million in 1998-99. 


19.71 The objective of the Research 
Partnerships Program is to foster 
interactions and partnerships between 
university researchers and other sectors in 
order to generate new knowledge and 
develop new expertise, and to transfer this 
new knowledge and expertise to 
Canadian-based organizations. 


19.72 Inrecent reports to Parliament, 
NSERC explains that one of its main goals 
is to facilitate the transfer of knowledge 
from universities to other sectors, and the 
commercialization of university-derived 
technologies. NSERC does this because 
university “partnerships with industry 
connect researchers with those who can 
use the new knowledge productively and 
thereby enhance Canada’s capacity for 
innovation.” In Part II of its 1997-98 
Estimates, NSERC reported that it 
facilitates knowledge transfer from 
universities to other sectors, and the 
commercialization of university-derived 
technologies. Its financial support for 
industry-university partnerships leads to 
new products, processes and services. 


19.73 Putting this program rationale 
into practice, NSERC set the following 
criteria for deciding on which research 
proposals to fund through the Research 
Partnerships Program: 


¢ merit of the research proposal 
(originality and quality of the research, 
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Exhibit 19.6 


Grants Under the Research 
Partnerships Program 


quality of the research team, benefits of 
the proposed research and its potential 
impact on Canada’s economy, industry, 
society and/or environment); 


e interactions and partnerships 
(nature and extent of contributions from 
participants, ability of the partners to 
exploit the research results to the benefit 
of Canada); 


e training (extent to which all 
participants are involved in the training of 
highly qualified personnel); and 


e management and budgeting ( funds 
requested from NSERC and contributions 
from other sources and its management). 


Although the Research Partnerships 
Program is made up of several different 
types of grants (see Exhibit 19.6), these 
criteria apply to all of them. In practice, 
the emphasis and extent of analysis by 
criterion depends on the category of grant. 


19.74 For our audit, we selected a 
sample of 100 grants from the various 
categories of grants made from 1994 to 
1997 under the Research Partnerships 
Program as well as seven Networks of 
Centres of Excellence grants made in the 
years 1994 to 1998. 


Concerns identified in support for 
funding decisions 


19.75 We expected that NSERC’s 
spending decisions would be based on all 


of the criteria it has set for its Research 
Partnerships Program. Because of the 
differences in the type of research and 
development projects funded — basic to 
pre-commercial — we also expected that 
the extent of and support for analysis of 
each criterion would vary. Nevertheless, 
due diligence requires that all of the 
criteria be appropriately considered. 


19.76 We found that the scientific merit 
of the projects we examined had been well 
established, as had the potential for 
training. However, scientific merit deals 
with the originality and quality of the 
proposed research, not with its potential 
significance to industry or other partners, 
or with innovation capacity building. 
There was often little information on file 
explaining the significance of the 
expected benefits to the partner company, 
or the need for government support. In our 
view, proper examination of all of these 
issues is essential to determining whether 
a prospective project is likely to enhance 
industry’s capacity for innovation. 


19.77 Scientific merit well evaluated. 
The merit of the research proposals and 
the quality of the researchers were well 
evaluated. NSERC uses a system of 
project review committees consisting of 
researchers and industry representatives. 
In addition, projects are submitted to 
external referees who provide a written 
evaluation of each proposal. The use of 
external referees greatly expands the 
expertise available within the panel and 


Strategic grants support high-quality pre-competitive university research that, if successful, will 
produce a specific economic, social, industrial or environmental benefit to Canada. Participation 


from outside the university is essential. 


Research Networks grants fund large-scale, complex research proposals that involve 
multi-sectorial collaborations on a common research theme and that demonstrate the added 
advantages of a networking approach. These networks involve at least five researchers from three 


organizations that are not formally affiliated. 


University-Industry grants support partnerships between industry and university that promote 
high-quality research of economic or industrial importance. The University-Industry projects 
include the following grants: Collaborative Research and Development, Industrially Oriented 
Research, Industrial Research Chairs, New Faculty Support and Chairs in the Management of 


Technological Change. 
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thus provides a vital source of information 
in the decision-making process. 


19.78 Little information on 
significance of expected results. Given 
NSERC’s criteria for funding and its 
statements to Parliament on the role of the 
program in developing partnerships that 
lead to new products, processes and 
services, we expected that funding 
decisions would take into account these 
kinds of expected results. Although the 
extent of this analysis would vary with the 
type of grant, we expected that the 
justification for funding a project would 
explain the usefulness and potential 
significance of the research results in 
pre-commercial and commercial terms. 


19.79 — For about a third of the projects 
we audited, we could not assure ourselves 
that NSERC had adequately considered 
the significance of the research for the 
partners. While the link between the 
research and the company’s operations 
was explained, the significance or extent 
of the potential benefit was not evident. 
For these grants, there was little 
explanation of how the projects might lead 
to significant new or improved products or 
processes or how much the firm’s 
knowledge and skills would be improved. 
For the remainder of the grants we 
audited, these issues were adequately 
considered. 


19.80 © NSERC has informed us that its 
project selection committees consider the 
expected benefits of the project to the 
partners, but this is not always 
documented. In our view, due diligence 
requires that the assessment of funding 
criteria be consistently documented so as 
to provide proper justification for 
decisions to spend public money and so 
that subsequent management review and 
performance measurement are possible. 


19.81 Little explanation of potential 
socio-economic benefits of proposed 
projects. Beyond their impact on 
individual firms, research projects should 
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have the potential to generate 
socio-economic benefits such as 
enhancing the country’s economic 
strength and developing new industries. In 
general, little information was provided on 
the potential socio-economic benefits of 
the projects other than the training of 
qualified personnel and the creation of 
direct jobs. While this assessment 1s 
obviously difficult for certain types of 
research grants, NSERC has not 
developed specific indicators or other 
decision aids to help program officers and 
project selection committees to reasonably 
assess whether proposed projects are 
likely to achieve the desired results. 
Where on file, the descriptions of possible 
economic benefits were very broad in 
nature and not substantiated. 


19.82 No evidence that the need for 
funding was considered. We expected 
that the project files would provide an 
explanation of why program funding was 
needed for the project to proceed. We 
found that the need for funding was not 
addressed in almost 90 percent of the 
projects. 


19.83 Few agreements on intellectual 
property rights. One of the funding 
criteria for these grants 1s that the partners 
or other potential users must have the 
capacity to apply the research results in 
Canada. To avoid potential conflicts, 
NSERC recommends that the partner 
firms and the universities involved in the 
research proposals negotiate a research 
agreement before starting a research 
project. Usually, the university and/or the 
researchers employed by the university 
own the rights to the intellectual property 
arising from NSERC grants. 


19.84 Intellectual property agreements 
would not be appropriate for all types of 
grants. NSERC recommends, but does not 
insist, that these agreements be negotiated 
before starting a research project. 
Nevertheless, we found that NSERC was 
often unaware of whether an intellectual 
property agreement had been reached. 
This finding points to a lack of attention 


For one third of the 
projects we audited, 
the significance or 
extent of the potential 
benefits to the 
partners was 

not evident. 


For almost 90 percent 
of the projects we 
audited, the need for 
government funding 
was not explained. 
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Performance 
information says 
little about key 
results expected of 
the Research 


Partnerships Program. 


to how projects will enhance Canada’s 
innovation capacity. 


Limited performance information 


19.85 As part of our audit, we 
examined whether management knows if 
the program is performing well. While the 
intended benefits and results of some of 
these projects may take time to realize 
fully, there are nonetheless a number of 
immediate benefits from NSERC support. 
These programs encourage university 
research and the advanced training of 
graduate students. They also increase the 
collaboration between the university, 
government and the private sector on a 
project-by-project basis. Reports to 
Parliament show that the number of firms 
involved with these programs has 
increased over time, as has the amount of 
money leveraged. Nevertheless, this 
performance information says little about 
performance in relation to some of the key 
results expected of the Research 
Partnerships Program. 


19.86 We noted the following 
limitations in measuring the results 
achieved by the Research Partnerships 
Program. 


19.87 Little information on impact of 
projects. NSERC regularly receives 
progress reports and final reports from the 
research partners. These reports indicate 
whether the project’s research and 
development goals are met; however, 
there is little information in the project 
files to indicate how the results of the 
research and development might be used 
to achieve commercial benefits such as 
patents, new processes or services or 
licences. Management informed us that in 
many of these cases, it is premature to 
assess possible benefits at the end of the 
project. However, in our view, project 
results need to at least be compared 
against the results expected when the 
project was approved and the likely 
pre-commercial and commercial benefits. 


19.88 Inthe sample of 100 files we 
examined, 11 projects had failed for 
various reasons while 31 were still 
ongoing and the remaining 58 had more or 
less achieved their objectives. However, 
we were unable to determine from 
NSERC’s records whether the research 
results were used by industry partners 
following the completion of the projects. 


19.89 One formal evaluation done. 
NSERC recently evaluated the 
performance of one type of grant under 
the Research Partnerships Program — its 
support for strategic projects. This 
evaluation found that, in order for the 
program to remain relevant, management 
should choose a clear role for it. As well, 
funded applicants indicated that many 
projects would not have proceeded 
without program funding. The evaluation 
also noted that little information on the 
results at either the program or the project 
level was available to management. The 
contribution of program funding to 
industrial, economic, social and 
environmental impacts was difficult to 
isolate from other effects. The evaluation 
recommended that the program objectives 
be more clearly defined and that 
performance indicators be used. 


19.90 A pilot study was conducted in 
1998 to measure some of the outcomes of 
the Collaborative Research and 
Development grants. This was not a full 
evaluation, as it did not address all the 
required evaluation issues. The study was 
done through a survey and indicated that 
while some industrial collaborators 
created new processes, products, standards 
or services, others updated their 
knowledge and had access to new ideas as 
a result of the funded research projects. 
The outcomes of many other projects were 
still unknown at the time of the survey. 


19.91 The Natural Sciences and 
Engineering Research Council should: 


e establish clear expected results for 
the Research Partnerships Program and 
report performance against them; 
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e ensure that the decision to fund a 
project is based on an appropriate 
assessment of the nature and 
significance of the project benefits and 
the need for NSERC support and is 
appropriately documented; 


e ensure that reliable information is 
gathered on the results of projects, 
including the benefits to industrial 
partners; and 


¢ conduct a program evaluation of 
the Research Partnerships Program 
that addresses all of the key evaluative 
questions. 


Natural Sciences and Engineering 
Research Council’s response: Around the 
world, research is recognized as 
fundamental to building a successful 
innovation system. In Canada, 
university-based research is particularly 
important because the research capacity 
of both the private sector and governments 
is limited relative to other G—7 countries. 
NSERC supports both basic university 
research through research grants and 
project research through partnerships of 
universities with industry, as well as the 
advanced training of highly qualified 
people in both areas. The chapter 
examines one dimension of NSERC 
investments, the Research Partnerships 
programs. We are happy that the chapter 
concludes that “these programs encourage 
university research and the advanced 
training of graduate students” and “they 
[the Research Partnerships programs] 
also increase the collaboration between 
the university, government and private 
sector”, thus achieving the goals and 
objectives of the Research Partnerships 
programs. 


NSERC shares the Auditor General’s goals 
of improving accountability and 
performance measurement. In that spirit, 
we accept the recommendations of this 
chapter. However, we feel obliged to raise 
a point about the context of the chapter. It 
must be remembered that university 
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research in partnership with industry is 
very different from industrial research, 
where clearly defined commercial benefits 
can be identified. In the Research 
Partnerships Program, NSERC invites 
industry to share in the risk of conducting 
university research that is long-term and 
often without quantifiable expected 
results. The expectations of results, 
documentation and follow-up must take 
this into consideration, and they must be 
commensurate with the small size of the 
investment. For example, a typical project 
in the Research Partnerships Program is 


funded at less than $100,000/year and can 


be as low as $5,000, a very small fraction 
of the funding of a Network of Centres of 
Excellence. These awards go only to 
universities, and in response to 
applications that succeed in a 
peer-reviewed competition, where 
applications are assessed against criteria 
that include the potential for impact of the 
research results on Canada’s economy, 
industry, society and/or environment. 
Many applications are turned down 
because they do not meet a high enough 
standard on these criteria. The chapter 
acknowledges that innovation, how it 
works, and how its performance is 
measured are very complex issues that are 
still not fully understood. Within this 
context, NSERC agrees that greater efforts 
are required to track the longer-term 
impacts of our investments, and therefore 
we are continuously improving 
performance measurement activities as 
our knowledge and understanding 
improve. 


Networks of Centres of 
Excellence 


19.92 The Networks of Centres of 
Excellence (NCE) program is managed 
jointly by NSERC, the two other granting 
councils (Social Sciences and Humanities 
Research Council and the Medical 
Research Council) and Industry Canada. It 
focusses on building strong links among 
university, government and industry 
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researchers working in different 
disciplines and widely separated 
institutions, and on accelerating the 
transfer of new technology to the private 
sector. The objective of the program is to 
improve Canada’s performance in science 
and technology, and to facilitate transfer 
of knowledge to those who can use it to 
advance Canada’s social and economic 
development. 


19.93 In addition to our audit criteria, 
we assessed the NCE grants using the five 
selection and evaluation criteria for the 
program, namely: excellence of the 
research program, highly qualified 
personnel, networking and partnerships, 
knowledge exchange and technology 
exploitation, and network management. 


19.94 We concluded that due diligence 
had been exercised in the grants we 
audited under the Networks of Centres of 
Excellence. We found that for each 
proposal submitted, there was a complete 
description of the network and its planned 
research activities, and the files included 
all relevant information. Each proposal 
was assessed by an expert panel and 
examined by the selection committee for 
final approval by the steering committee. 


19.95 All the networks provide regular 
reports on the progress of the projects and 
site visits are done. Every agreement 
defines intellectual property ownership. 


19.96 Finally, a 1997 evaluation 
concluded that the Networks of Centres of 
Excellence program had succeeded in all 
of its objectives. While the report did not 
consider all of the basic evaluation issues, 
it suggested that the program will provide 
substantial net economic benefits. 


Technology Partnerships Canada 


19.97 Technology Partnerships Canada 
(TPC) was created in 1996 as a special 
operating agency within Industry Canada. 
The program is intended to promote the 
development and commercialization of 
innovative technologies that contribute to 


increasing economic growth and creating 
jobs and wealth. 


19.98 TPC has been directed to take an 
investment approach, sharing risks as well 
as returns, with all repayments being used 
to fund future investment opportunities. 
Repayments are normally based on 
royalties on sales. On average, the TPC 
sharing ratio is not expected to exceed 

33 percent of eligible project costs. Some 
highly successful projects may return 
much more than the original investments 
to help offset less successful projects. 
TPC’s annual budget is $300 million. 


19.99 To be eligible for funding, 
projects must be in a qualifying 
technology sector: environmental 
technology, enabling technology (that 1s, 
advanced manufacturing and processing 
technology, advanced materials, 
biotechnology and selected information 
technology) and aerospace and defence 
industries. TPC has been mandated to 
target one third of its funding to 
environmental and enabling technology. 


19.100 Between 1996 and the end of 
June 1999, TPC had made 81 
contributions for a total of $827 million. 
We audited 30 contributions, representing 
$580 million and ranging from $161,000 
to $100 million. 


19.101 We expected TPC to ensure that 
the projects represented value for money. 
To do this, funding decisions need to be 
based on and respect all the assessment 
criteria set for the program by the 
Treasury Board Secretariat, and should 
achieve a reasonable balance in meeting 
all of them. 


Due diligence in assessing business case 
issues 


19.102 We found that funding decisions 
adequately considered, and achieved a 
reasonable balance among, the following 
commercial issues: 


e the significance of the proposed 
innovation to the firm and to industry; 
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e the expected project benefits to the 
firm; 


e the recipient’s ability to carry out the 
project and exploit the results 
commercially; and 


e the likelihood of success. 


19.103 We concluded that TPC had 
exercised due diligence in assessing 
whether these projects represented value 
for money for the applicant to carry out. 


Support for funding decisions 


19.104 We found that TPC’s project 
assessments adequately addressed: 


e applicant eligibility; 
e the need for TPC funding; 


e the expected leverage of new 
investment; and 


e the net expected economic and other 
benefits. 


However, we have the following concern. 


19.105 Little support for amount of 
contribution to three projects. The 
amount of TPC’s contribution is supposed 
to be based on an assessment of the 
minimum funding required by the 
applicant to proceed with the project and 
on the forecasted benefits for Canada. 


19.106 We found that TPC had assessed 
the minimum amount required by 
applicants for all the projects we audited, 
except in the cases of three large 
contributions. For these contributions, 
management claims that the justification 
for the amount of government funding 
included a challenge to the cost of the 
project, analysis of the project’s 
socio-economic benefits, and historical 
knowledge of the industry. Based on this 
analysis, TPC concluded that the amount 
was the minimum required to make sure 
the project proceeded in Canada. 
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19.107 Nevertheless, we expected that 

the specific amount of the contribution 

would be supported by an analysis of the 

company’s ability to carry out the project 

without support, and the minimum TPC 

investment needed to make the project 

successful. Because TPC had not carried x ; 
out this analysis for these agreements, we So far, little attention 
are concerned that it did not assure itself has been given to 
that the specific amounts of these he : 
contributions were the minimum needed monitoring projects 


for the projects to proceed. and results. 


Performance monitoring and reporting 


19.108 TPC is a relatively new program 
and management has spent the first years 
establishing due diligence practices for 
assessing proposed projects. Now that 
these practices are in place, management 
is turning its attention to improving 
project monitoring. 


19.109 Project and results monitoring 
could be improved. We expected that 
TPC would monitor the progress of 
funded projects and ensure that the 
applicant is meeting all its obligations 
under the funding agreement. 


19.110 TPC assures itself prior to 
payment of a claim that funds are being We concluded that 
used for the purposes agreed. We noted Technology 

that TPC monitored the progress of : 

projects through some site visits and that Partnerships Canada 
it had conducted a few audits to date. had exercised due 
However, TPC does not yet systematically diligence in assessing 
monitor project results. For example, 
recipients are required to send the business cases for 
semi-annual and annual reports. Those the projects. 

reports are supposed to contain 
information on progress and the project’s 
contribution to jobs and economic growth. 
TPC does not ensure that all the reports 
are received on time and there is little 
information on actual jobs created. 
Although we found audited financial 
statements on file, there was no evidence 
that such information was reviewed on an 
ongoing basis to help assess the continued 
viability of the companies or the projects. 
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Reporting to Parliament 


19.111 We examined TPC’s reporting to 
Parliament. Since TPC supports 
longer-term projects and it will take time 
to achieve substantial results, its reporting 
to date has been on projected 
performance. 


19.112 Leveraged funds need to be 
explained. TPC has reported that during 
the first two years of its operations, it 
invested $563 million of public funds that 
would leverage $2.4 billion of new private 
sector investments. For about 

$400 million of the new private sector 
investment, we found that TPC had also 
counted other government funding, federal 
and provincial research and development 
income tax credits, investments prior to 
project funding and possible future 
investments in related projects, as well as 
money spent outside the country. TPC 
ought to disclose the sources and 

uses of leveraged funds to avoid 
misinterpretation. 


19.113 Reporting to Parliament needs 
clarification. We examined how the 
program’s approach to making 
contributions is described in reports to 
Parliament. Under its terms and 
conditions, one of TPC’s objectives is to 
manage its contributions by taking an 
investment approach through sharing in 
returns as well as risks. 


19.114 Reports to Parliament have 
repeated this performance expectation. 
For example, Industry Canada’s 
1999-2000 Report to Parliament on Plans 
and Priorities states, ‘“The assistance is 
fully repayable, with TPC sharing in both 
the risks and rewards of the projects with 
its partners.” TPC’s most recent annual 
report states that “the program supports 
the private sector through investment 
rather than subsidy, sharing in both risks 
and rewards.” 


19.115 However, we found that the 
reports do not explain how management 
interprets these aspects of TPC’s mandate. 


In particular, there is no explanation of the 
basis for or extent of sharing risks and 
returns with firms. In practice, how TPC 
shares risks and returns varies 
significantly among agreements. Under 
many agreements, TPC will recover its 
contribution through royalties on all sales 
of the supported product or technology, 
starting with the first sale. Under other 
agreements, TPC assumes more risk by 
starting to receive royalty payments only 
after several years of sales or after a 
certain number of products are sold. For 
still other agreements, royalty payments 
are structured in such a way that TPC’s 
contributions are unlikely to be fully 
repaid. 


19.116 Management has indicated that 
specific repayment terms are negotiated 
on a case-by-case basis and are arrived at 
by assessing applicant need, firm and 
project risk and the magnitude of the 
strategic benefits that would result if the 
project is undertaken in Canada. 
Management also informed us that it must 
balance the objective of “taking an 
investment approach through sharing in 
returns as well as risks” with other TPC 
program objectives such as increasing 
economic growth and wealth creation. 


19.117 Nevertheless, in our view, proper 
reporting to Parliament requires that TPC 
explain how it interprets and acts on key 
parts of its mandate. In particular, the 
meaning of “taking an investment 
approach through sharing in returns as 
well as risks” and the extent to which 
contributions are “fully repayable” need to 
be better explained in any report to 
Parliament. 


19.118 Industry Canada should ensure 
that: 


e there is appropriate justification 
for the specific amount of all 
contributions; and 


e the performance of projects funded 
by Technology Partnerships Canada is 
appropriately monitored. 
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In its reports to Parliament, Industry 
Canada should: 


e explain the sources and uses of 
funds leveraged by its contributions; 
and 


e provide information to clarify: 


— the extent to which TPC shares 
risks and returns; 


— the basis on which TPC shares 
risks and returns, including 
the main factors taken into 
consideration when 
establishing royalty payments; 
and 


— the extent to which 
contributions are “‘fully 
repayable”. 


Industry Canada’s response: The 
Department notes that the Auditor 
General concluded that Technology 
Partnerships Canada exercises due 
diligence in assessing projects. One key 
aspect in this regard is the assessment of 
the minimum amount of assistance 
required to make sure that projects 
beneficial to Canada proceed. The 
information set available to make such a 
determination varies from project to 
project and consistently requires the 
application of considerable judgment. As 
explained in paragraph 19.106, 
management is of the opinion that it does 
take all reasonable measures to ensure 
that there is appropriate justification for 
the specific amount of all contributions. 
The few specific projects that gave rise to 
the Auditor General’s observation do not 
follow the traditional nature of product 
development assistance. TPC, therefore, 
developed alternative evaluation 
strategies in support of their assessment. 
The Department believes this approach 
was and is appropriate for these projects. 


As indicated in the chapter, TPC is a 
relatively new program and management 
has appropriately concentrated on, first, 
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establishing rigorous due diligence 
processes for assessing proposed projects 
and, second, the effective management of 
the processing of claims. Management will 
now focus on ensuring that funded 
projects are appropriately monitored. To 
this end, all investments will be reviewed, 
client reporting requirements brought up 
to date, and a comprehensive portfolio 
monitoring system implemented before 
31 March 2000. 


In reporting leverage, TPC’s intent is to 
communicate additional levels of 
innovation spending generated as a result 
of TPC investments. Therefore, TPC 
leverage data focus on the use rather than 
the source of spending. In the future, TPC 
will more fully describe the basis on which 
it reports leverage so as to ensure a more 
comprehensive understanding. 


TPC’s objective is to invest to maximize 
the number of projects at the minimum 
level of support required, and hence also 
to maximize the private sector investment 
leveraged. In some circumstances, this 
results in accepting greater repayment risk 
in order to ensure that projects proceed. 
The framework within which TPC 
operates is complex and requires judgment 
and trade-offs to arrive at the best mix of 
risks as well as rewards. Given the public 
policy mandate of the program, its risk 
and reward sharing is not restricted to 
merely the financial aspects of the 
investments. The Department will expand 


future reporting to be included in TPC’s 


Annual Report, which is tabled in 
Parliament. 


Conclusion 


19.119 We examined whether 
management of the programs we audited: 


e could show that these programs are 
designed to improve Canada’s innovation 
performance; 


e exercised due diligence in investing 
in innovation projects; and 
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e knew if the programs were achieving 
value for money. 


19.120 We found that management of 
each program is developing goals and 
measures for assessing performance. 
However, the expected results for these 
programs are not expressed in innovation 
performance terms. Clear statements of 
expected results are prerequisites to sound 
program design and management, and to 
measuring and interpreting the actual 
results achieved. 


19.121 We concluded that there are 
significant opportunities for management 
to improve the exercise of due diligence in 
approving contributions under the 
Industrial Research Assistance Program 
(IRAP) and grants under the Research 
Partnerships Program. 


e We found that only about 15 percent 
of the funding decisions under IRAP were 
based on an assessment of all of the 
project criteria, mainly because the need 


for IRAP support had not been considered. 


We also found a range of practices in how 
the other criteria were assessed — in 
particular, how projects would improve 
the technological capability of recipients. 
In our view, the assessment of individual 
funding criteria for many projects was not 
sufficiently thorough. 


e For the Research Partnerships 
Program, we found that the scientific 
merit of the projects we examined had 
been well established, as had the potential 
for training. However, scientific merit 
deals with the originality and quality of 
the proposed research, not with its 
potential significance to industry or other 
partners, or with building innovation 
capacity. We found that there was often 
little information on file explaining the 
significance of the expected benefits to 
the partner company, or the need for 
government support. 


19.122 For both IRAP and the Research 
Partnerships Program, we found important 
performance issues for which management 
had little information. 


e We noted that various efforts have 
been made to assess the performance of 
IRAP services. Nevertheless, management 
needs to gather reliable information on the 
results of funded projects and its advisory 
services, and to investigate the extent to 
which its funding supports projects that 
would not have proceeded otherwise. In 
our view, any job creation numbers 
reported for the program need to be 
carefully explained and qualified. 


e We also noted that the information 
gathered on the results of projects funded 
under the Research Partnerships Program 
is limited to success in relation to research 
goals. There is little information on the 
success of the projects relative to the 
commercial and pre-commercial benefits 
expected when they were approved. As 
well, only one part of the program had 
been recently subjected to a program 
evaluation. 


19.123 We concluded that due diligence 
had been exercised in the grants we 
audited under the Networks of Centres of 
Excellence program. A 1997 program 
evaluation concluded that the program had 
succeeded in all of its objectives. 
Although the report did not consider all of 
the basic evaluation issues, it suggested 
that the program will provide substantial 
net economic benefits. 


19.124 We also concluded that 
management of Technology Partnerships 
Canada had exercised due diligence in 
making the contributions that we audited, 
with some specific exceptions. 
Nevertheless, project and results 
monitoring and reporting to Parliament on 
expected performance need to be 
improved. 
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y About the Audit 


Objectives and Scope 


Our audit covered four programs in the Industry Portfolio — the Industrial Research Assistance Program 
(IRAP), the Research Partnerships Program, the Networks of Centres of Excellence and Technology 
Partnerships Canada (TPC). 


Specifically, we examined whether management of the programs we audited: 
e could show that these programs are designed to improve Canada’s innovation performance; 
e exercised due diligence in investing in business innovation; and 


e knew if the programs were achieving value for money. 


Approach 


We examined project files, analyzed data and documentation and conducted interviews with officials for each 
of the selected programs. We reviewed relevant literature and studies, and also did some site visits on a 
sample of projects. 


Criteria 


We expected that management would assure itself that each project represents value for money (for the 
applicant to carry out) by determining: 


e the significance of the innovation; 

e the expected project benefits; 

e the recipient’s capability to carry out the project and exploit the results successfully; and 
e the Itkelihood of success. 


We expected that management would assure itself that the project represents value for money (for the 
program to fund) by determining: 


e—eligibility; 

e compatibility with program objectives and existing portfolio of grants or contributions; 
e need for assistance/expected leverage; 

e net expected economic and other benefits; and 

e = likelihood of repayment (TPC). 


We expected that management would exercise due diligence in providing advice to meet client information 
needs (IRAP). 
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We expected that management would assure itself that the program is achieving expected results by assessing 
whether : 


project goals are met; 


e expected project benefits are realized; 

e expected net economic benefits are realized; 

e conditions of funding are respected; 

e repayments owed the Crown are made (TPC); and 
@ advice meets clients’ needs (IRAP). 

Audit Team 


Assistant Auditor General: Richard Flageole 
Principal: Peter Simeoni 
Director: Rafid Warsalee 


Isabelle Barolet 
Louise Bertrand 
lan Campbell 
Veronique Dallaire 
Bryan De Pape 
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Julie Pelletier 
Michelle Salvail 


For information, please contact Peter Simeoni. 
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Fisheries and Oceans 


Pacific Salmon: Sustainability 
of the Fisheries 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Canadian Institute of Chartered 
Accountants. The numbered paragraphs in bold face represent recommendations. 
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Fisheries and Oceans 


Pacific Salmon: Sustainability 
of the Fisheries 


Main Points 


20.1 The Pacific salmon fisheries are in trouble. Catches have declined overall, but the commercial catch has 
plummeted. The long-term sustainability of the fisheries is at risk because factors like overfishing, habitat loss, 
and declining ocean productivity have eroded the resource base. The result is a fisheries management crisis that 
has cast a cloud of uncertainty over the future of the salmon fisheries. Salmon fishing will continue, but more 
stringent controls are needed in the short term to ensure that salmon survive for the benefit of future generations. 


20.2 The management challenge for the Department of Fisheries and Oceans is to conserve existing stocks and 
rebuild those that are at low levels, while maintaining the viability of the fisheries. It will have to adapt its 
management regime to the new realities and gain the acceptance and support of stakeholders if it is to be 
successful. These will be difficult tasks to accomplish. 


20.3 The Department now needs to: 
e identify information needs and priorities; 
e review the consultation process and identify areas for improvement; 
e implement integrated fisheries management plans; 
e complete the downsizing of the fleet as a priority; 
e establish an independent allocation board; and 


e develop a better working relationship with the Province of British Columbia where fishery issues are 
concerned. 


Background and other observations 


20.4 The Department has already taken the first steps to address the challenges it faces. It has affirmed 
conservation as its primary objective to protect existing salmon stocks and rebuild the resource base. In 1998 it 
announced a new policy, A New Direction for Canada’s Pacific Salmon Fisheries, to direct its management 
activities in the future. The regional office has developed and is implementing its strategic plan for 1998-2001 
based on this policy. 


20.5 In addition to the management of larger stocks targeted primarily by the commercial fishery, the regional 
office has begun to focus more on the conservation of smaller stocks that are important to the Aboriginal and 
recreational fisheries. It has taken a conservation-based approach to fisheries management and is testing selective 
fishing methods and gear, in addition to implementing special programs such as fleet reduction and restructuring. 
However, there are areas that need attention if policy objectives are to be attained. The most important are 
improving regional databases and information management to enable it to manage Pacific salmon on a 
stock-by-stock basis, and resolving consultation problems to improve stakeholder relations and move toward 
forming partnerships to share management responsibilities and offset costs. 


20.6 The Department will need time to adjust to the new management regime. During this time, fisheries will 
be subject to closure to allow stocks to recover. Improvements cannot be expected to occur quickly, given that 
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salmon life cycles can extend to five years or longer and salmon need time to adjust to changes in their freshwater 
and marine habitats. Finally, the future sustainability of the fisheries will depend not only on the Department’s 
ability to implement its New Direction policy but also on its success in enlisting the help of stakeholders and the 


provincial government to share the workload. 


The responses of Fisheries and Oceans to our recommendations are included in this chapter. The 
Department agrees with all 13 recommendations and notes that initiatives are under way in several areas. 
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Introduction 


20.7 Salmon are important to 
Canadians and, to British Columbians, 
serve as a powerful symbol of the quality 
of life. They generate a wide range of 
economic, social and cultural benefits for 
a broad spectrum of interests — the 
commercial and recreational fishing 
sectors together with the processing and 
service industries, First Nations, the 
federal and provincial governments, 
environmental groups and the public. 


20.8 Pacific salmon are caught in 
commercial, recreational and Aboriginal 
fisheries by a variety of methods and gear. 
Commercial fishers use seine and gill nets 
or lures drawn along behind the boat 
(trolling). Recreational fishers use rod and 
line; Aboriginal fishers use various means 
including dip nets, gillnets and traps. 
Exhibit 20.1, Glossary, defines these and 
other terms used in this chapter. 


Gear Types in the Commercial Fishery 
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20.9 Five species of salmon under 
federal management are taken in the 
fisheries — sockeye, pink, chum, chinook 
and coho. They have similar life histories 
in that all use both freshwater and marine 
habitats and all die after spawning. Each 
species comprises a number of distinct 
populations or stocks, which are spread 
throughout the streams, rivers and lakes of 
British Columbia. These stocks have 
different spawning areas and spawn at 
different times; the juvenile salmon that 
result use different feeding and rearing 
areas. 


20.10 Such differences are inherited 
and have produced a wide diversity of 
stocks, each adapted to local conditions. 
This genetic diversity provides a “safety 
net,” which is an important defence for 
salmon against the impacts of natural and 
man made changes to their environment. 
While the strengths of individual stocks 
will vary over time according to the nature 
and location of such impacts, their 


Seine 
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20.11 The focus on biodiversity has 
occurred at a time when salmon fisheries 
in British Columbia are in serious trouble. 
Natural factors (such as the effects of 
global warming on marine and freshwater 
temperatures, and fluctuating ocean 
productivity); human factors (such as 


diversity enhances the ability of the 
species to respond to change and 
ultimately to survive. The conservation of 
diversity is an objective of the United 
Nations Convention on Biodiversity 
(1992), to be implemented through the 
Canadian Biodiversity Strategy (1995). 


Exhibit 20.1 


Glossary 


Abundance. Number of fish in a stock or 
fishery. 


Allocation. The number of fish assigned 
or allotted to a particular group or 
individual. 


Assessment. Evaluation of the 
productivity of a stock as a basis for 
deciding how many salmon should spawn 
and what harvest rates should be. 


Biodiversity (biological diversity). The 
variability among living organisms, 
including the terrestrial and marine 
ecosystems and the ecological complexes 
of which they are part. This includes 
diversity within species, among species 
and of ecosystems. See genetic diversity. 


Capacity (harvest). Related to the 
discussion of overcapacity. The 
Department defines “harvesting capacity” 
broadly to include the number of vessels 
in the fleet, the mobility of vessels, fishing 
technology, the quantity and type of gear 

| used, and the skill of individual vessel 
operators. 


Closure. Official end to a period of legal 
fishing. 


Co-management. Sharing of management 
responsibilities among two or more 
agencies or parties. 


Conservation. The management of 
salmon stocks to ensure that adequate 
numbers of salmon spawn each year, that 
spawning is successful, and that genetic 
diversity is maintained. 


Conservation unit. A group of one or 
more local populations that share a 
common genetic lineage and that can be 
managed effectively as a unit by virtue of 


their common productivity and 
vulnerability to existing fisheries. Similar 
to the evolutionarily significant units 
(ESUs) used for the U.S. Endangered 
Species Act. Distinct population segments; 
a geographic area containing genetically 
related or similar groups. 


Escapement. Number of mature salmon 
that pass through (or escape) the fisheries 
and return to their rivers of origin to 
spawn. 


Genetic diversity of salmon. The 
separation of individual stocks of a 
salmon species based on inherited 
characteristics. This results, for example, 
in adults using different locations and 
times for spawning. See biodiversity. 


Gillnet. A type of commercial fishing gear 
— a net set upright in the water to catch 
fish by entangling their gills in its mesh. 


Habitat. Area in which an organism 
would naturally be found; the place that is 
natural for the life and growth of an 
organism. 


Landing. Salmon caught in the 
commercial fishery that are delivered to 
fish processors. 


Observer. Enforcement staff placed on 
board fishing vessels to monitor by-catch 
and use of selective fishing techniques. 


Overfishing. Excessive fishing; fishing to 
depletion. 


Pacific Scientific Advice Review 
Committee (PSARC). The Committee 
provides internal and external clients with 
scientific information that is reliable, 
relevant, timely and comprehensive. It 
advises the Resource Management 


Executive Committee chaired by the 
Director General and other bodies (like 
the newly formed Pacific Fisheries 
Resource Conservation Council) on stock 
and habitat status and potential biological 
consequences of fisheries management 
actions and natural events. 


Precautionary approach. See Exhibit 
205: 


Risk-averse management. Management 
system weighted to conservation of a 
stock. 


Seine. A type of commercial fishing gear 
—a large fishing net with floats along the 
top edge and weights along the bottom. 


Selective fishery. Conservation-oriented 
management approach that allows for the 
harvest of surplus target species or stocks 
while aiming to minimize or avoid the 
harvest of species or stocks of 
conservation concern. 


Stock. Fish returning to a general 
geographical area for spawning. Part of a 
fish population whose use is under 
consideration. 


Sustainable development. Development 
that meets the needs of the present without 
compromising the ability of future 
generations to meet their own needs. 


Sustainable use. Management aimed at 
passing on a robust, healthy and abundant 
salmon resource for the enjoyment of 
future generations. 


Test fishing. Fishing activity designed to 
provide data from which an estimate of 
run size can be generated. 


Troll. A type of commercial fishing gear 
—a line trailed behind a moving boat. 
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overfishing and loss or deterioration of 
habitat); and economic factors (such as 
commercial fleet overcapacity and 
competition in the marketplace from a 
growing salmon-farming industry), are 
contributing to a decline in fishing 
opportunities, success rates, and value of 
the catch. The Department’s challenge is 
to conserve existing stocks and rebuild 
those that are at risk, while maintaining 
viable fisheries. 


Focus of the audit 


20.12 This is the second of three audits 
of Pacific salmon management. The first, 
reported in 1997, examined the 
Department’s management of the resource 
base, concentrating on the conservation of 
salmon habitat. The third audit will 
examine the Department’s role in the 
salmon-farming industry and the 
industry’s impact on the conservation, 
production and management of wild 
salmon. 


20.13 The objective of the current audit 
was to determine if the Pacific salmon 
fisheries are being managed to ensure the 
conservation of the resource base and the 
sustainability of the fisheries. The audit 
reviewed existing problems and actions 
taken by the Department to respond to the 
new management challenge in three major 
areas: policy development and planning, 
fisheries and information management and 
government-stakeholder consultations. At 
the request of the Public Accounts 
Committee, we also incorporated into the 
audit a follow-up of the Department’s 
response to the recommendations 
contained in our 1997 chapter, Fisheries 
and Oceans — Pacific Salmon: 
Sustainability of the Resource Base. 


20.14 Further details on the audit 
objective and criteria are found at the end 
of the chapter in the section About the 
Audit. 


Observations and 
Recommendations The Department's 
Challenge is to 
conserve existing 


salmon stocks. 


The Troubled Pacific Salmon 
Fishery 


Catches have declined 


20.15 Statistics show that the 
commercial catch has dropped 
dramatically and the recreational catch is 
declining; however, the Aboriginal catch 
appears to be relatively stable (see 
Exhibit 20.2). There has been a rapid drop 
in the value of commercial catches since 
1995 (see Exhibit 20.3). 


The resource base is being eroded 


20.16 The fisheries draw on a 
diversified resource base of over 8,000 
individual stocks of sockeye, pink, chum, 
coho and chinook salmon. A 1996 review 
of Fisheries and Oceans data, published by 
the American Fisheries Society, found that 
sufficient information was available to 
assess the status of only 60 percent of 
these stocks; the status of the remaining 
AO percent was undocumented. Of the 
stocks assessed, over 15 percent were in 
some degree of danger. Coho stocks were 
the most affected overall, with 20 percent 
at risk. 


20.17 In spite of the Department’s 
efforts, the fish habitat loss we reported in 
1997 is still occurring. This contributes to 
the continuing decline of many salmon 
stocks. 


20.18 A new factor has recently come 
to the attention of Fisheries and Oceans 
scientists — apparent changes in ocean 
productivity, which have resulted in 
unforeseen declines in the numbers of 
salmon returning to spawn. This has made 
it necessary to restrict or close fisheries to 
conserve stocks. The most recent closures 
occurred in August of this year in the 
Fraser River, when the regional office’s 
pre-season estimate of sockeye expected 
to return had to be adjusted from 

8.2 million to 3 million. 


Information is 
unavailable for 

40 percent of salmon 
stocks. 


Habitat loss is still 
occurring. 
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20.19 The availability of salmon to the 
fisheries has been affected by the ongoing 
dispute between Canada and the U.S. 
Pacific Coast states, especially Alaska and 
Washington, over their respective shares 
of salmon originating in British Columbia 
streams. Although Canada and the U.S. 
signed the Pacific Salmon Treaty in 1985 


Numbers Caught 
(millions) 


Exhibit 20.2 


45 


to deal with conservation and equity 
issues, from 1992 until recently they 
disagreed on annual fishing arrangements. 
In 1992, both countries began to fish coho 
heavily. Some progress toward resolving 
the dispute was made in 1998 when 
arrangements were negotiated with the 
northern U.S. states to share catches and 


Pacific Salmon Catch Data 
(1991-1998) 


Commercial ——————---—~ 30 < 


Aboriginal 


Recreational 


T 
Source: Compiled from statistics 1991 1992 1993 1994 


provided by Fisheries and Oceans and 
Edwin Blewitt and Associates 


($ millions) 
300 
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Note: Commercial, recreational and Aboriginal catch data after 1994 are preliminary. 
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Salmon Catch Values 250 
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exchange information on salmon stocks. 
Finally, in June 1999, a new agreement on 
conservation and equity issues under the 
Treaty was announced. The agreement 
provides for improved conservation and 
scientific co-operation. Bilaterally agreed 
fishing arrangements are now to be based 
on abundance instead of on numerical 
limits. The future impact of these 
arrangements on Canada’s fisheries has 
yet to be seen. 


20.20 Coho stocks declined 
dramatically in the 1990s as a result of 
overfishing, habitat loss and other factors. 
Stringent measures have become 


Fisheries and Oceans — Pacific Salmon: 
Sustainability of the Fisheries 


necessary to conserve these stocks for the 
long-term sustainability of the fisheries. 
The coho “story” is a good illustration of 
how the management crisis in which the 
Department finds itself developed (see 
Appendix A). 


Sustainability Will Depend on 
DI 
q 


Sound Planning 


The Department’s planning record at 
the national level is poor 


20.21 Inthe absence of a strategic plan, 
the Department has been struggling over 
the last decade to deal with threats to 


Examples of Recreational 
and Aboriginal Fisheries 


All photos courtesy of Fisheries and Oceans 
Ocean recreational fishery 


Aboriginal fishery — dip net 


Aboriginal fishery — beach seine 
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The regional office has 
developed a strategic 
framework for salmon 
management. 


biodiversity and declines in the resource 
base. Its record of strategic planning has 
been poor, as indicated in our reports and 
its own internal evaluations. The 
Department did not, until this year, have a 
national strategic planning framework in 
place that would allow it to respond in a 
coherent way to internal and external 
challenges. 


20.22 In October 1998, the Department 
finalized a departmental Business Plan for 
the period 1999-2001. The Business Plan 
provides a frank discussion of the 
challenges facing the Department, but 
lacks specific targets to be met in this 
period. For instance, the Department has 
not set priorities and targets for protecting 
the various salmon stocks. 


20.23 Fisheries and Oceans has now 
re-initiated strategic planning at the 
national level and has completed the first 
two stages of a department-wide strategic 
planning process. It has identified external 
and internal factors that are likely to affect 
its operations over the next two to five 
years and has developed an overview of 
the long-term strategic issues it faces. An 
integrated departmental strategic plan is 
being developed, with fall 1999 the target 
date for its completion. 


20.24 Fisheries and Oceans is 
spearheading a government-wide strategy 
for the protection of marine areas — the 
Oceans Strategy — which will integrate 
all ocean-related programs including fish 
management, science, enforcement, and 
habitat protection. This will add to the 
complexity of its strategic planning 
activities. 


20.25 The Department has also begun 
to develop strategic plans at the national 
level for each of the three operational 
sectors involved in salmon management. 
The plan for the Fisheries Management 
Sector has been stalled, however, while an 
overlapping of responsibilities with the 
newly formed Oceans Sector is sorted out. 
While no strategic plan has yet been 
completed for the Science Sector, some 


areas have been addressed — for example, 
data management. 


Implementing a new management 
regime 


20.26 The Department’s lack of 
follow-through on its national and sector 
planning processes forced its regional 
office to move forward to address urgent 
issues in the area of Pacific salmon 
management. The regional office 
developed its own strategic framework in 
the spring of 1998 — New Directions for 
Fisheries & Oceans in the Pacific: A 
Departmental Strategy for 199S8—2001 — 
in part to direct the Department’s focus 
away from crisis management. As well as 
contributing to implementing the Oceans 
Act, the strategic framework focusses on 
conservation and sustainable use of the 
resource, on changes in decision making 
(participation, involvement and 
co-operative management), and on 
communications. The framework 
highlights the need for an integrated 
policy to direct the Department’s actions. 
The regional office has since worked on 
delegating responsibility for activity areas 
under the strategic framework, and has set 
timelines for making progress in these 
areas. 


20.27 The strategic framework is being 
implemented, in part, through a new 
departmental policy announced in October 
1998 — A New Direction for Canada’s 
Pacific Salmon Fisheries (New Direction 
policy). The policy reconfirms the 
Department’s commitment to 
sustainability and biodiversity, and 
introduces new elements into the 
management regime (see Exhibit 20.4). It 
is important to note that it relies heavily 
on a sound science base to drive decision 
making. 


20.28 The New Direction policy builds 
on work begun earlier on coho salmon 
conservation, fleet reduction and 
restructuring, selective fishing, habitat 
conservation and protection, and 
economic adjustment. Many of the 
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Department’s requirements for sustainable 
fisheries as stated in the policy have been 
validated by experts in both the public and 
private sectors (see Appendix B). 


20.29 The New Direction policy sets in 
motion significant changes in the way the 
Department will do business in the future: 
it identifies three objectives for salmon 
fisheries management — conservation, 
sustainable use and improved 
decision-making — together with 
principles to guide the policy’s 
implementation. 


20.30 The New Direction policy 
identifies clear goals and guidelines for 
developing operational policies and 
thereby provides a framework for 
measuring the Department’s future 
performance in: 


e protecting genetic diversity of 
stocks; 

e meeting the scientific and 
operational requirements of the 


precautionary approach (explained in 
Exhibit 20.5); 


e conserving the productive capacity 
of habitat; 


¢ applying selective fishing techniques 
to protect weaker stocks; and 


e involving stakeholders and the 
provincial government, in decision 
making, while moving toward the ultimate 
objective of fisheries co-management 
through the use of regional management 
boards. 


20.31 The Department has moved 
forward, releasing discussion papers on 
allocation and selective fishing and asking 
the public, governments and stakeholders 
to comment on them through a 
consultative process. Discussion papers on 
salmon management and improved 
decision making are being prepared for 
release before the end of the year. 


20.32 The regional office has 
completed an Integrated Regional 
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Business Plan for 1999-2000 — the first 
major integrated planning effort under the 
strategic framework and a response to the 
New Direction policy. The regional office 
has informed us that there were extensive 
consultations between national and 
regional headquarters during the 
development of this business plan. It 
remains unclear, however, how the 
Department’s New Direction policy, 
together with the strategic framework and 
business plan of the regional office, will 
be integrated with or affected by its 
national strategic plan and the Oceans 
Strategy. 


20.33 The regional strategic framework 
creates a made-in-British Columbia 
solution to Pacific regional problems in 
the context of national priorities. It 
recognizes that the regional office must 
address problems of internal integration 
— including a lack of integrated program 
delivery at the local level — and develop 


The New Direction 
policy provides an 
accountability 
framework for 
assessing 
departmental 
progress. 


Operational policies 
will be developed in 
consultation with 
stakeholders. 


Exhibit 20.4 


Elements of A New Direction for Canada’s Pacific Salmon Fisheries 


Old Principles Restated 


New Principles Added 


diversity 


e Net gain in productive 
capacity of habitat 


Conservation e More emphasis on genetic 


] 
z | 
e Precautionary approach | 


e Ecological approach 


Nations 


Sustainable Use e Priority allocation to First 


e No trade offs 

|e Selective fishing 

|e Recreational fishery’s 
priority for coho and | 
chinook 

e A diversified and viable 
commercial fishery 


Improved Decision 


Making public 


e Improved stakeholder 
consultation process/ 
structure 


Provision of information to 


Shared accountability with | 
stakeholders 


|e Input to decision making 


|e Proposed 


regional/watershed 
management boards 


| 
i 


Source: Fisheries and Oceans. A New Direction for 
Canada’s Pacific Salmon Fisheries. October 1998. 
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plans for organizational changes to 
address them. 


20.34 ‘Internal co-ordination problems 
have impeded policy development and 
planning. The Department recognizes that 
there has been a lack of policy 
co-ordination between regional and 
national headquarters because of differing 
priorities and that this poses a threat to the 
effectiveness of its salmon management 
policies. The downstream effects are 
problems in integrating internal planning, 
communicating a clear and consistent 
Future success in message, evaluating results, and building 
evaluations into the planning cycle. The 
salmon management Department appears to be making progress 


will depend ona sound __ in integrating national and regional 
; policies. 
science base. 
The New Management Challenge 


Biodiversity-based conservation 


20.35 Regional managers developed 
fisheries management regimes in the past 
that focussed on the largest or most 
productive stocks in the Fraser and Skeena 
River systems. Now the regional office 
has begun to widen its management focus 
to include smaller stocks in order to 
protect biodiversity. 


20.36 Given the need to satisfy 
conservation requirements while 
optimizing fishing opportunities, a better 


Exhibit 20.5 


Precautionary Approach to the Management of Fisheries 


e Protect living marine resources and preserve the marine environment. 
e Be more cautious when information is uncertain. 


e Improve decision making by sharing best scientific information and adopting 
ways to deal with risk and uncertainty. 


e Adopt stock specific reference points for conservation (limits of exploitation) 
and for fisheries (targeted catch levels). 


e Take into account uncertainties (all types), as well as impact on non-target 
species. 


Source: Chapter 14 of the 1997 Report of the Auditor General — Fisheries and 
Oceans — Sustainable Fisheries Framework: Atlantic Groundfish 


understanding of the genetic diversity of 
stocks is essential. The focus on 
biodiversity and the example of the coho 
crisis have served to illustrate the 
shortcomings of current management 
systems, procedures and practices. The 
success of the new salmon management 
regime under the New Direction policy 
will depend largely on the scope and depth 
of the Department’s science base. 


20.37 The regional office has indicated 
that to protect genetic diversity it will 
manage salmon on the basis of 
conservation units — groupings of stocks 
with related genetic characteristics — 
similar to those adopted by the U.S. It has 
done some work on major stocks and has 
developed proposed conservation units for 
coho, but much remains to be done before 
the stocks of all five salmon species are 
classified in this way. Fisheries 
management plans will need to recognize 
these conservation units and ensure their 
continued integrity. 


20.38 To protect the genetic diversity 
of salmon stocks, Fisheries and Oceans 
should move quickly to determine 
conservation units for all five species. 


Department’s response: Fisheries and 
Oceans agrees with the need to continue 
efforts to determine conservation units for 
Pacific salmon. Work on coho salmon 
stocks is most advanced at this point, 
reflecting immediate conservation 
concerns. Initial plans for conservation 
units for all species will be completed in 
priority sequence, as quickly as resources 
permit, then continually upgraded as new 
information becomes available. 


20.39. To evaluate the status of 
conservation units, managers and 
stakeholders will need comprehensive, 
integrated and understandable reports on 
the status of both stocks and habitat. The 
reports published by the U.S. on 
threatened salmon stocks in the northern 
states are good examples. Alternatively, 
the regional office could pattern its status 
reports on the format developed by the 
Committee on the Status of Endangered 
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Wildlife in Canada (COSEWIC). Its 
reports would then likely meet the 
requirements under proposed endangered 
species legislation for registering stocks at 
risk, and would form the basis of detailed 
plans for recovering those stocks. 


20.40 The Department should 
produce comprehensive, integrated 
status reports on stocks and habitats 
based on the new conservation units for 
each salmon species. The reports should 
be updated annually and used in 
developing, implementing and 
evaluating fisheries management plans. 


Department’s response: There is a need to 
improve the integration of stock and 
habitat assessment information to help 
guide fisheries management decisions. 
This is consistent with the ecological 
approach to fisheries management to 
which the Department is committed, and 
will be implemented in a staged manner. 
The Department agrees that integrated 
reports should be produced on a regular 
basis, with more frequent reviews in 
special circumstances, but it questions 
whether annually is the appropriate time 
frame for regular reporting. 


Basic information on stocks and 
habitats is incomplete 


20.41  Daia on escapement and catches 
are vital information for assessing the 
status of stocks. To date, however, the 
regional office’s data collection has been 
weighted toward the larger stocks 
originating in the major salmon 
production areas (for example, Skeena 
coho, Fraser sockeye and pink salmon). 


20.42 A 1996 report by the American 
Fisheries Society indicated that a 
significant lack of information exists on 
the status of Pacific salmon stocks. The 
report reviewed the status of over 8,000 
sockeye, pink, chum, coho and chinook 
stocks in British Columbia, based on 
available escapement data (that is, 
information on the number of salmon that 
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arrived on the spawning grounds). There 
was not enough information to assess the 
status of 40 percent of those stocks. A 
continuous record of stock assessment 
data is available on larger stocks 
traditionally used in commercial fisheries 
and on many stocks in the Fraser and 
Skeena River systems. However, basic 
information on coho is limited to 
approximately 50 percent of known 
stocks, and almost no information is 
available on the smaller stocks of all 
species. 


Stocks will be 
managed within 
conservation units. 


20.43 The regional office’s Salmon 
Escapement Database is a primary source 
of information to assess the strength of a 
stock. Along with reliable or scientifically 
supported observations, it contains a 
mixture of unqualified estimates of 
escapement that require expert 
interpretation. Data therefore cannot be 
accessed directly by the user, who must go 
through the Database manager. The 
inaccessibility and incompleteness of the 
data limit their usefulness to fisheries 


Good escapement and 
catch data are vital for 


managers, and preclude easy access by ion 
stakeholders. This database has remained Coneee 
essentially unchanged for 20 years, but the management. 


regional office now has plans to develop a 
new escapement database that will 
classify data by level of quality and 
provide Internet access. 


20.44 = There are data on the 
productivity of freshwater habitats but not 
enough on those used by individual stocks 
to provide a comprehensive picture of 
their health. Reporting on stock and 
habitat status is neither comprehensive nor 


Spawning salmon 


Photo courtesy of 
Fisheries and Oceans 
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Recent developments 
in Canada-B.C. 
relations are 
encouraging. 


Improvements to 
databases are being 
initiated. 


integrated, which poses problems for the 
regional office in protecting the genetic 
diversity of salmon stocks. 


20.45 In addition, there is a lack of 
historical information on relationships 
among habitat status, habitat change and 
fish production. This prevents experts 
from being able to define the productive 
capacity of habitats, and to protect critical 
habitats. 


20.46 Despite the Department’s efforts, 
the loss of fish habitat is contributing to 
the continuing decline of many salmon 
stocks. The Canada-British Columbia 
Agreement on the Management of Pacific 
Salmon Fishery Issues (1997) has 
generated collaborative efforts at the 
working level in several areas, including 
habitat protection, but progress has been 
limited. Efforts by the two governments to 
produce standardized information on 
habitat inventory are continuing, as we 
noted in 1997. Stull, significant 
improvements in the habitat information 
database likely cannot be expected for 
some time. A positive development in this 
area was the May 1999 signing of the 
Sub-agreement to the Canada-British 
Columbia Agreement on the Management 
of Pacific Salmon Fishery Issues 
Respecting Fisheries Information 
Co-ordination and Sharing. 


20.47 In following up on our previous 
recommendations, we noted that the 
Department has made progress in habitat 
protection. In 1998 the Department and 
British Columbia signed an agreement 


Commercial fishing: 
seine boats 


Photo courtesy of 
Fisheries and Oceans 


respecting work on coastal and marine 
areas. However, it is still in the early 
stages of such initiatives as regional and 
community-based planning. 


20.48 The regional office has since 
begun to develop a process of assessing 
the extent of habitat loss but has not yet 
completed the framework for carrying out 
the review. In the meantime, it has 
announced the hiring of 15 Habitat 
Auxiliary staff to help in monitoring 
habitat. 


20.49 Improvements are needed in the 
collection of catch data. Commercial 
vessels make approximately 70,000 
landings annually, and fish can be sold 
fresh to buyers up and down the coast. 
Large processing companies supply 
generally reliable sales-slip data 
electronically. However, it is difficult to 
obtain similar data on salmon sold directly 
to the public by fishers or small operators. 


20.50 Fishers recognize the 
Department’s problem. They believe that 
independent, accurate catch monitoring 
and observer programs are critical to any 
future fishery under the management 
regime established by the New Direction 
policy. 


20.51 It is an offence under the 
Fisheries Act to fail to provide catch data 
or to provide inaccurate catch data. In 
areas that are not frequented by fisheries 
officers, fishers are required to report their 
landings to the nearest departmental office 
by phone. However, the regional office’s 
1998 report on phone-ins by commercial 
fishers revealed low compliance for seines 
(49 percent) and gillnets (43 percent). 


20.52 An onboard observer program to 
document numbers of coho captured and 
released was initiated in the 1998 fishing 
season. According to the regional office’s 
data, however, in some areas only 

2 percent to 13 percent of the total 
boat-days for sockeye and chum were 
monitored. Further, a lack of coverage by 
observers of packer vessels collecting fish 
from commercial seine fishing boats 
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created problems in getting accurate 
counts of coho salmon taken in the 
fishery. Seiners account for as much as 
50 percent of the coho taken accidentally 
while fishing for other species. 


20.53 A recent Supreme Court ruling 
upheld the requirement that the 
recreational fishing industry report catch 
data, and this should improve compliance 
in the future. In 1998 the regional office 
observed a mixture of good and bad 
reporting of catches by sport lodges and 
charter operations. Furthermore, it 
believed that in some cases recreational 
fishers underreported their encounters of 
coho by about half the actual number. We 
found that the regional office survey 
coverage of the recreational fishing effort 
was limited overall, which raises 
questions about the accuracy of the data. 


20.54 Managing for biodiversity will 
create a new set of problems in data 
collection. More precise catch data will be 
needed in 2000 to manage fishing 
opportunities in special management 
zones, where controlled fishing can occur. 
Under these conditions, in-season 
sampling of numbers of returning salmon 
(test fishing) is important. In 1998, 
management of the commercial fishery 
relied heavily on test fishing results. 


20.55 The Department has 
acknowledged that further improvements 
in its catch, escapement and habitat 
databases are needed. The Data 
Management section of the Department’s 
Science Strategic Plan 2005 initiative 
called data an “extremely valuable and 
irreplaceable resource”’, and stated that 
“data accessibility and integration have 
become serious concerns.” The Plan 
raised such needs as data centres for 
regional directors, adequate resources for 
data management, and new technology for 
distributing data internally and externally. 


20.56 The Department should assess 
its information requirements in the 
areas of data collection, analysis, and 
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management, in order to meet its 
long-term needs and identify priorities 
under the New Direction policy. 


Department’s response: Fisheries and Catch reporting 
Oceans concurs with this remains a problem. 
recommendation, and is preparing 
assessment frameworks for all species of 
Pacific salmon. These frameworks will 
define the information required to ensure 
conservation and effective management, 
and will be used to determine priorities 
for allocation of resources under the New 
Direction policy. The Auditor General 
cites some progress under the 
Canada-—British Columbia Agreement on 
the Management of Pacific Salmon 
Fishery Issues. This work has already 
resulted in significant improvements in 
salmon information management, and 
work to achieve further improvements will 
continue. 
Managing for genetic 
The Department is not making best use diversity will require 


of the Aboriginal Fisheries Strategy 
(AFS) better data. 


20.57 The AFS has assumed a major 
role in data collection. The Department 
relies on Aboriginal communities to 
collect escapement and other data. In fact, 
figures obtained from the regional office 
indicated that 30 percent of new data 
being input to the Salmon Escapement 
Database come from this source. Through 
the AFS, moreover, First Nations are 
developing a pool of expertise that will 
assist the Department in managing salmon 
resources as land claims settlements are 
negotiated. 


20.58 The AFS contributes to 

knowledge of salmon stocks through a The Aboriginal 
variety of fisheries management activities, Fisheries Strategy 
including catch monitoring and stock and : ; 
habitat assessment. Aboriginal plays a major role in 
conservation officers (fishery guardians), data collection. 
who are mainly involved in enforcement 

activities, receive formal training in data 

collection. However, usually only 

on-the-job training is provided to 

technicians and stream-walkers, who are 
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Data quality could be 
improved. 


The regional office’s 
process for evaluating 
scientific advice is 
Open and transparent. 


Spawning salmon 


Photo courtesy of 
Fisheries and Oceans 


largely responsible for stock and habitat 
assessment. 


20.59 The Department acknowledges 
the value of involving Aboriginal peoples 
in monitoring, research, enhancement and 
habitat conservation, but it has concerns 
about the quality of data provided under 
some agreements. We found that it has no 
clear system for collecting information 
from First Nations and forwarding it to the 
Science Branch for compilation. It also 
lacks uniform standards to ensure that 
information from different sources can be 
compared. This has resulted in significant 
gaps in the data, and problems with their 
reliability and timeliness. 


20.60 At the time of our audit the 
Science Branch had received catch data 
for 1997 from fewer than 15 percent of the 
bands that were required to collect it. In 
1998, the regional office reported that 
some First Nations on the north coast 
submitted either no catch data or data that 
were unusable. The regional office told us 
that some bands refuse to provide catch 
reports on salmon because of 
complications in treaty negotiations. We 
also learned that the involvement of 
Fisheries Officers in obtaining data from 
bands and submitting them to the Science 
Branch is highly inconsistent across 
districts. The regional office is now 
working to update its reporting systems 
and to better delineate the responsibilities 
of everyone involved in the process. 


20.61 The Department should 
evaluate the comprehensiveness and 
quality of data collected under the 
Aboriginal Fisheries Strategy (AFS) and 
the adequacy of the standards and 
procedures that guide data collection, 
compilation and reporting, with a view 
to improving and expanding the role of 
the AFS in this area. 


Department’s response: The role of First 
Nations, through the Aboriginal Fisheries 
Strategy, in data collection and reporting 
is evolving. First Nations are becoming 
more proficient at collection and reporting 
of data. The Department acknowledges the 
need to more rigorously define data 
quality standards and methods, and to 
establish reporting procedures. Fisheries 
management staff are working with the 
Science, Stock Assessment and Habitat 
and Enhancement branches to integrate 
the process of collecting and reporting the 
data. 


Improved process for reviewing 
scientific information and advice 


20.62 Fisheries managers receive 
advice from the regional Pacific Stock 
Assessment Review Committee — 
recently renamed the Pacific Scientific 
Advice Review Committee (PSARC). 
This committee is the regional body 
responsible for the review and evaluation 
of all scientific information on salmon 
stocks and their habitats. This information 
is critical to the development of 
sustainable fishing plans. 


20.63 However, the PSARC does not 
deal specifically with traditional 
knowledge — the knowledge acquired 
over time by participants in the 
Aboriginal, commercial and recreational 
fisheries. Yet the Department does 
recognize this as an important 
consideration in the development of 
fishing plans to ensure that all relevant 
information has been taken into account. 


20.64 Scientific information undergoes 
peer review to ensure its reliability, and 
advice generated by this review is 
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forwarded uncensored to senior 
management. PSARC reports are made 
available to the public. The process is 
both open and transparent, and 
stakeholders can participate in it. 


20.65 = The PSARC was restructured in 
1999, when a Habitat Sub-Committee was 
added. PSARC is now required to provide 
reports on stock status, although only on 
the major stocks, and in the future it will 
also issue reports on habitat status. 
However, there is no requirement to 
integrate the two reporting responsibilities 
on a stock-by-stock basis. PSARC’s stock 
status reports are public documents that 
summarize, in lay terms, scientific and 
technical information on major 
commercially harvested species. In our 
opinion, more comprehensive reports are 
needed on individual stocks, or on groups 
of stocks within the proposed conservation 
units, to facilitate salmon fisheries 
management under the New Direction 
policy. 


20.66 The Department should ensure 
that the responsibilities of the Pacific 
Scientific Advice Review Committee are 
in line with the needs outlined in A New 
Direction for Canada’s Pacific Salmon 
Fisheries, by: 


e requiring the Committee to 
produce comprehensive integrated 
reports on stock and habitat status, 
taking into account traditional 
knowledge; and 


¢« expanding the Committee’s area of 
reporting to cover individual salmon 
stocks or groups of stocks under 
proposed conservation units. 


Department’s response: Fisheries and 
Oceans agrees with this recommendation. 
The Department will be moving to ensure 
that PSARC salmon stock status reports 
incorporate habitat status information. 
The Committee is already responsible for 
incorporation of traditional knowledge in 
its assessments, and for reporting on the 
status of individual stocks or groups of 
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stocks. Stock status reports will be aligned 
with conservation units, once defined. 


Toward integrated fisheries 
management planning 


20.67 Management strategies and 
policies, together with the information on 
stock assessments, are translated into 
practical actions through annual fishing 
plans. The regional office’s present fishing 
plan process is based on a salmon 
management model that has remained 
essentially unchanged for many years. 
There are ingrained problems in the 
existing system. Over 10 years ago, in our 
1986 Report, we noted that the regional 
office’s work plan did not co-ordinate 
science, habitat and enforcement activities 
to ensure that they were consistent with 
the regional office’s objectives. In 1998, 
the Department’s Review Directorate 
identified continuing problems among 
departmental sectors in co-ordinating and 
following the fisheries management 
planning process in all regions. 
Post-season evaluations of fishing plans 
have not been documented. Currently, the 
incorporation of any lessons learned into 
the next year’s fishing plans depends on 
the continuity of the people involved. 


Better reporting on the 
status of stocks and 
habitat is needed. 


The importance of Integrated Fisheries 
Management Plans 


20.68 The Department is committed to 
moving from the existing fishing plan 
process to a long-term planning process 
for salmon management. Integrated 
Fisheries Management Plans (IFMPs) 
were not implemented for salmon in 1998, 
despite the Department’s intentions to do 
so. However, in its Integrated Business 
Plan for 1999-2000 the regional office has 
indicated that one salmon IFMP will be 
developed in 1999. The Department has 
used IFMPs in other fisheries since 1996, 
but has had difficulty using them in 
salmon fisheries. The IFMP process 
integrates the activities and specialized 
knowledge of every departmental sector 
involved, and uses broader consultation 
with stakeholders beyond those with direct 


Integrated Fisheries 
Management Plans are 
essential to success. 
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The Department has 
adopted the 
precautionary 
approach to salmon 
management. 


At present, recovery 
plans for threatened 
stocks are not 
mandatory. 


fishing interests. The goal of the process is 
to produce fishing plans that are more 
comprehensive and longer-term than the 
annual plans currently in use. In our 
opinion, IFMPs are essential to the 
successful implementation of the new 
management regime. 


20.69 IFMPs will incorporate the 
precautionary approach, which is an 
internationally accepted approach to 
managing declining fisheries. It provides 
direction to managers who have limited 
scientific information, while recognizing 
the need to expand the knowledge base 
(see Exhibit 20.5). The Department has 
not yet decided how it will apply this 
approach to managing the salmon 
fisheries. A critical element of IFMPs is 
the development of stock reference points 
that indicate acceptable catch levels 
(targets) and levels beyond which no 
fishing will occur (conservation limits). 
Targets will be influenced and held in 
check by economic considerations, but the 
conservation limits will have to be rigidly 
enforced if biodiversity is to be protected. 
While the available information on some 
stocks 1s sufficient to set such limits, there 
is little or no information on the majority 
— especially smaller stocks. Under the 
precautionary approach, closures may 
therefore be necessary to protect 
individual stocks while their status is 
determined. 


20.70 In 1998, the Department 
announced its plan for the recovery of 
coho stocks. However, the IFMP process 
at present does not require comprehensive 
recovery plans for threatened stocks, 
although they will likely be mandatory 
under proposed endangered species 
legislation to meet the requirements of the 
Canadian Biodiversity Strategy. 


20.71 The Department should ensure 
that Integrated Fisheries Management 
Plans include formal recovery plans for 
stocks at risk. 


Department’s response: The Department 
agrees in principle with the 


recommendation to include formal 
recovery plans for stocks at risk in 
Integrated Fisheries Management Plans. 
We are developing a coho plan for 
recovery of upper Skeena and Thompson 
coho. Fishery restrictions were introduced 
starting in 1998 that curtail harvest by all 
sectors, involving fishery closures and 
adjustments to the area, timing and gear 
specifications of permitted fisheries. 
Salmon habitat improvement initiatives 
have also been authorized and projects in 
support of selective fishing practices have 
been conducted. The Department will 
develop recovery plans consistent with 
these specifications when available, and 
include them in IFMPs. 


20.72 The Department should 
facilitate the application of the 
precautionary approach to salmon 
fisheries management by establishing 
catch levels and conservation limits for 
individual stocks or groups of stocks. 


Department’s response: This 
recommendation is consistent with the 
Wild Salmon Policy now under 
development by the Department. The 
Policy, which is based on the 
precautionary approach, will establish 
escapement levels and target harvest rates 
that will ensure long-term sustainability. 
This work goes hand-in-hand with the 
requirement to establish conservation 
units and will be a central feature of 
departmental science input to fisheries 
management. 


The use of selective fishing methods 


20.73 Selective fishing methods will be 
an important element of the new 
Integrated Fisheries Management Plans. 
Salmon are often harvested in mixed-stock 
fisheries, where fish from different stocks 
and species intermingle. Selective fishing 
limits the harvesting of weaker stocks, 
while allowing the harvesting of stronger 
stocks by requiring non-targeted salmon to 
be released. Thus, conservation is 
promoted while fishing 1s allowed to 
continue. Selective fishing is recognized 
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internationally as a management tool to 
promote sustainable fisheries. 


20.74 The February 1999 Selective 
Fisheries Multi-Stakeholder Workshop 
report provided information on the results 
of selective fishing to date. The 
effectiveness of selective fishing in 
reducing the mortality rates of fish taken 
accidentally and then released has yet to 
be proved. For example, the one 
experiment that took place in the Alberni 
Inlet in September 1998 raised questions 
about the long-term survival of released 
salmon. Short-term mortalities were low, 


Selective Fishing Methods 


Catch and release (recreational) 


All photos courtesy of Fisheries and Oceans 
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but departmental data on long-term 
mortality rates was inconclusive. The 
Pacific Scientific Advice Review 
Committee raised the concern in 1995 that 
selective fishing relies on uncertain and 
still unproved technologies. We are 
concerned that, to date, there is still no 
conclusive evidence that most of the 
released salmon survive to spawn. 


20.75 In 1998, based on short-term 
mortality rates, the regional office 
estimated coho mortalities at 93,000 in all 
fisheries. However, if significant numbers 
of released salmon are in fact dying before 


Brailing (seine) 
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Data on long-term 
mortality rates of 
released salmon are 
inconclusive. 


Fleet overcapacity has 
been a problem for 
more than a decade. 


they reach the spawning grounds, this 
estimate would be too low, with severe 
consequences for the conservation of the 
species. 


20.76 In many British Columbia 
recreational fisheries, there is a lack of 
information on the effects of fishing gear 
and methods on short- and long-term 
mortality rates. There is also little 
information on the survival of coho that 
are hooked and then released. Without this 
information, it is difficult to assess the 
impact of recreational fisheries on the 
conservation of weak stocks. However, we 
acknowledge that the collection of data on 
long-term mortality in the fisheries may 
be both difficult and costly. 


20.77. The Department should assess 
the risks to conservation of allowing 
selective fishing in the commercial and. 
recreational fisheries, given the lack of 
reliable information on long-term 
mortality rates of released salmon. It 
should then build adequate safeguards 
into fishing plans to protect stocks at 
risk. 


Department’s response: The Department 
is continuing studies to improve 
understanding of the mortality of salmon 
released following capture in commercial, 
recreational and First Nations fisheries. 
The knowledge gained through these 
studies will be incorporated into future 
fisheries management plans. Current 
management plans take account of 
expected mortalities based on existing 
knowledge. 


Reducing the size of the commercial 
salmon fleet 


20.78 Fleet overcapacity has had an 
impact on both the conservation and the 
economic sustainability of the commercial 
fishery. Fisheries and Oceans had been 
advised of overcapacity problems in the 
salmon fishery for over a decade before 
taking action under the Pacific Salmon 
Revitalization Strategy in 1996. The 


20-22 


initiative introduced several strategies for 
decreasing harvesting capacity, such as 
reducing the size of the fleet through a 
voluntary vessel buy-back, and reducing 
the mobility of vessels and quantity of 
gear through changes in licensing for 
areas and gear. The Department aimed to 
reduce fleet capacity by 50 percent over 
the long term. However, it could not 
adequately measure the effects of this 
reduction on fish catches; instead, it 
measured the numbers of licences it 
retired or the number of vessels fishing in 
a given area. This meant that it could not 
evaluate the Strategy’s contribution to 
conservation objectives or determine what 
further restructuring would be necessary 
to achieve them. 


20.79 The restructuring continued in 
1998 under the Pacific Fisheries 
Adjustment and Restructuring (PFAR) 
Program. By the summer of 1999, two 
rounds of licence buy-backs had been 
conducted — one in the fall of 1998 and 
one in the spring of 1999. At least one 
more round of buy-backs was planned for 
the fall of 1999, when fishers would have 
more complete information about the 
Department’s plans for the fishery of the 
future and its new management regime. 
The first two rounds of buy-backs offered 
compensation in line with what was 
offered in 1996, and almost the same 
number of licences were retired: 746 in 
fall 1998/spring 1999, compared with 797 
in 1996 (see Exhibit 20.6). There are now 
2,557 licences remaining in the salmon 
fleet that are eligible for buy-back. 


20.80 The Department has committed 
itself to reducing capacity in the seine, 
gillnet and troll fisheries, thus conserving 
stocks, reducing dependency on the 
resource, and increasing the economic 
viability of the industry. But it has not 
measured its progress toward the first two 
outcomes, and has therefore set no 
conservation-based targets for fleet 
reduction. However, it has set specific 
targets based on calculations of the licence 
reductions needed to create the conditions 
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for the commercial fishing industry to 
become economically viable. 


20.81 Changes in the fishery and in the 
Department’s management approach have 
implications for the size of fleet that 
would be economically viable. The 
Department based its licence buy-back 
target on calculations that reflected its 
risk-averse management strategy. 
However, these calculations did not 
anticipate any significant changes in fish 
harvesting or management, such as the 
potential for widespread closures where 
salmon from weaker stocks cannot be 
sufficiently avoided or released unharmed 
through selective fishing. 


20.82 The Department’s intention to 
transfer management costs to the 
commercial fishery depends on the 
fishery’s economic viability. Fishers must 
first be earning enough income to support 
themselves and the activities that are 
required to manage the resource 
effectively. While the Department’s 
economic target for the buy-back initiative 
under PFAR was reasonable given the 
status quo, the target is not in keeping 
with current management changes. 
However, the buy-back attempts to 
address this problem by encouraging 
fishers to expand their interests beyond 
salmon. 


20.83 As well as providing a more 
sustainable livelihood for those who 
remain in the fishery, the buy-back is 
meant to enhance the prospects for 
achieving salmon conservation objectives. 
But the Department remains unclear on 
the extent of reductions needed. 
Furthermore, it has not yet integrated the 
buy-back program with other fleet 
management programs, such as selective 
fishing. However, in spite of the problems 
that we have indicated, fleet reduction and 
restructuring are helping the Department 
to move toward a sustainable commercial 
fishery. 


20.84 The Department should specify 
a fleet reduction target and timetable 
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that are consistent with its objectives of 
conservation, selective fishing and cost 
recovery, and work to complete fleet 
reduction according to this timetable. 


Department’s response: The Department in the number of 
agrees with this recommendation. In 1996, licences has been 
a multi-year salmon fleet reduction target : 

of 50 percent was established. This target achieved. 

will be reviewed taking into account 

various factors, in particular the 

requirement to fish selectively in order to 

meet conservation objectives, and harvest 

diversification opportunities. 


A significant reduction 


Allocation issues persist 


20.85 The absence of an allocation 
policy has created problems in relations 
between the Department and fishers. 
Before 1990, the Department had no 
formal policy for the allocation of fish to ‘ 
various users. In that year the Supreme Allocation has been a 
Court of Canada, in the Sparrow decision, perennial problem. 
affirmed the constitutional right of 

Aboriginals to fish for food, including fish 

consumed for social and ceremonial 

purposes. The Court held that the 

Department had to give such fisheries top 


Exhibit 20.6 


Pacific Commercial Fleet Reduction Since 1996 
Percentage 

100- — 
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50- 
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Seine Gillnet Troll Total 


i Eligible licences remaining in the salmon fishery 


[=] Pacific Fisheries Adjustment Restructuring Program reductions — fall 1998, spring 1999 


__| Pacific Salmon Revitalization Strategy reductions — 1996 


Source: Fisheries and Oceans 
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Problems with 
stakeholders are not 
new. 


priority after implementing valid 
conservation measures. No action was 
taken to resolve the issue of allocations to 
the commercial and recreational fisheries 
until 1998, when the New Direction policy 
was announced. 5 


20.86 In conjunction with the policy, 
the Department released in December 
1998 An Allocation Framework for 
Pacific Salmon 1999-2005, a discussion 
paper that proposes to formalize changes 
in the allocation process and to establish 
an independent allocation board. The 
board would advise the Minister of 
Fisheries and Oceans and assist him in 
implementing the allocation policy in an 
open and transparent way. This would 
likely remove a major source of friction 
between the Department and the 
stakeholders. 


20.87 The Department should act on 
its proposal to establish an independent 
allocation board as soon as possible. 


Department’s response: The Department 
agrees and is developing an 
implementation plan for the establishment 
of an allocation board. The final draft of 
An Allocation Policy for Pacific Salmon 
will be released soon. It provides for the 
establishment of an allocation board and 
outlines the board’s basic goals. 


The Support and Participation of 
Stakeholders Are Essential 


20.88 The Department sees consultation 
as a first step toward a closer working 
relationship with its stakeholders, as 
documented in its Framework and 
Guidelines for Implementing the 
Co-Management Approach. The New 
Direction policy commits the Department 
to broadening and enhancing input to 
decision-making, through regional boards. 


20.89 The Panel Studying Partnering, 
appointed by the Minister of Fisheries and 
Oceans, highlighted issues surrounding 
current relationships with stakeholders. 


The Panel found that even though the 
Department was making strong 
commitments to partnering, in practice it 
was still taking a “business-as-usual, 
top-down approach to fisheries 
management’. It concluded that the 
Department was “trying to deal with too 
many issues on the back of partnering” 
and pointed to the challenges of 
overworked staff, limitations in 
stakeholder capacity, and the time and 
energy involved in negotiating 
agreements. Because of the Department’s 
poor record of transparency in its 
consultations, stakeholders are wary about 
the possibility that it will give preferential 
treatment to certain interests; they are also 
skeptical about the workability of 
partnerships in fisheries management. 


Consultation problems have not been 
addressed adequately 


20.90 Reports going back as far as 
1982, when the Commission on Pacific 
Fisheries Policy issued Turning the Tide, A 
New Policy for Canada’s Pacific 
Fisheries, have documented 
dissatisfaction with the Department’s 
consultation process. In 1998, the process 
was Still being criticized in reports such as 
the West Coast Report by the Standing 
Committee on Fisheries and Oceans. 


20.91 The regional office made 
promises throughout the 1990s to improve 
its consultations and its consultation 
process, but it has not yet done so. The 
regional office did take steps to strengthen 
its relationships with stakeholders in 
developing the 1998 salmon fishing plans, 
but problems remain. Although the 
regional office consults extensively on the 
development of fishing plans, the way it 
participates in the consultation process has 
been criticized: there is a perception that 
the Department has already decided on the 
fishing plans before consultations begin. 
While this criticism could be a reflection 
of flaws in the consultation process, it 
could also indicate disagreement with the 
outcome. 
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20.92 The regional office acknowledges 
its ad hoc approach to consultations. It 
notes that consultation structures are not 
well integrated or well designed to deal 
with the full range of issues involved in 
managing the salmon fisheries. 
Shortcomings in its consultations on 
annual fishing plans and on such larger 
issues as restructuring and allocations 
have added to the Department’s loss of 
credibility. 


20.93 Several studies of the 
Department’s interactions with 
stakeholders and of stakeholder 
satisfaction with its consultation processes 
have noted the following widespread 
problems: 


e exclusion of stakeholders from 
consultations: 


e dissatisfaction with the transparency 
of the overall process; 


e failures to notify stakeholders about 
consultations, and holding consultations 
too late to affect decision-making; and 


e stakeholders’ perceptions that their 
input is not reflected in decisions and 
subsequent actions. 


20.94 From a list of 82 advisory groups 
that the regional office provided, we 
contacted 41 groups by telephone to see if 
they were satisfied with current 
consultation methods. Only 6 of the 
groups (15 percent) said they were. We 
did not look at the regional office’s 
consultations with Aboriginal advisory 
groups, with whom the Department has a 
legal obligation to consult. 


20.95 Although stakeholders continue 
to complain, the regional office has 
completed no evaluation of the 
consultation process. It has not developed 
a complete list of stakeholder contacts, 
and is not clear on whether various groups 
play an advisory or a consensus-building 
role. It is therefore not surprising that the 
process is not as effective as it could be. 
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In fact, regional managers told us that 
they think they consult too much. 


20.96 We recognize that it is not always 
possible to reach agreement with 
stakeholders on issues that are 
contentious. Sometimes the Department 
may have to take action without 
consulting further. When it does, however, 
we would expect it to give stakeholders an 
opportunity to discuss how to minimize 
such situations in the future. 


Shortcomings in the 
present consultation 
process have been 
documented. 


20.97 If its relationship with 
stakeholders does not improve, the 
Department will find it difficult to move 
forward with partnership arrangements 
and co-management, as discussed in its 
Framework and Guidelines for 
Implementing the Co-Management 
Approach and its New Direction policy. 
Part of the Department’s plan for 
partnering is to transfer some costs of 
fisheries management to users of the 
resource. For instance, it expects salmon 
harvesters to assume by 2001 the full costs 
of testing and experimenting with new 
selective fishing gear, and to eventually 
bear the costs of all catch monitoring and 
reporting. 


20.98 The Department has recognized 
the need for a fundamental review and 
revision of its present approach to public 
involvement in salmon fisheries 
management. As a first step, it plans to 
soon release a discussion paper on 
improved decision-making that will 
propose improvements to the consultation 
process. This will give it an opportunity to 
continue its efforts to regain public 
confidence. The improved decision 
making framework it proposes includes 
five elements — a Pacific Fisheries Policy 
Forum, an independent Pacific Fisheries 
Allocation, Licensing and Appeal Board, 
updated advisory processes for salmon 
management, ideas for the potential role 
of Area-Based Multi-Stakeholder Boards, 
and a Consultation Secretariat to support 
the decision-making process. It is an 
ambitious framework, proposing changes 


We confirmed a high 
degree of stakeholder 
dissatisfaction. 
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Improved relations 

with the Province of 
British Columbia is 

essential. 


Spawning salmon 


Photo courtesy of 
Fisheries and Oceans 
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to stakeholder involvement in advisory 
roles and partnering arrangements. 


20.99 The Department should 
evaluate its consultation process, with 
the input of stakeholders, to identify 
where improvements are needed before 
it finalizes its improved decision-making 
policy. 


Department’s response: Consistent with 
this recommendation, the Department is 
planning to obtain stakeholders’ and 
public input on how to improve the 
consultative process before finalizing the 
improved decision making policy, which is 
being drafted. 


Canada-British Columbia relations 


20.100 It is important for the Department 
to have a good working relationship with 
its major partner, the Province of British 
Columbia. Canada and the Province have 
overlapping responsibilities for the 
fisheries. Under the Canada-British 
Columbia Agreement on the Management 
of Pacific Salmon Fishery Issues, the two 
governments agreed to work together on 
Pacific salmon fishery issues, with the 
Province involved primarily in the area of 
habitat protection. 


20.101 In view of the provincial 
government’s role in habitat protection 
and its interest in promoting economic and 
cultural well-being, its co-operation and 
support are crucial. The Agreement calls 
for them to form joint advisory and 
administrative bodies, yet each party has 
acted unilaterally to establish its own (the 


Department’s Pacific Fisheries Resource 
Conservation Council and the Province’s 
Fisheries Renewal B.C.). While there is 
co-operation at the working level, the two 
governments have not yet agreed on joint 
goals for fisheries management that 
incorporate both conservation and 
economic objectives. Problems with 
implementing the Canada-B.C. 
Agreement have made the management of 
the resource base more difficult. However, 
the recent Agreement on Interjurisdiction- 
al Cooperation with Respect to Fisheries 
and Aquaculture signals an improvement 
in Canada—British Columbia relations. 


20.102 The Department should 
intensify its efforts to develop common 
objectives and integrated strategies with 
the Province of British Columbia to 
conserve the resource base and promote 
sustainable fisheries. 


Department’s response: The federal and 
B.C. governments have been working 
actively and jointly to implement the 
Canada-British Columbia Agreement on 
the Management of Pacific Salmon 
Fishery Issues. /n the two and a half years 
since the agreement was signed, the two 
governments have fulfilled many of the 
requirements of the agreement. Additional 
work on co-ordinating our efforts in 
enforcement and habitat management and 
restoration are under way. Also, new areas 
for co-operation are constantly under 
consideration. 


Identifying costs and setting priorities 


20.103 There is a gap between the 
Department’s high-level policy 
commitments to sustainable development, 
biodiversity conservation and stakeholder 
partnerships under the New Direction 
policy and its ability to meet those 
commitments. The Department continues 
to make strong claims about what it is 
trying to accomplish. For example, it sees 
itself as working toward achieving “the 
highest possible standards of scientific 
excellence” and achieving “no net loss of 
habitat’. Yet the Department has 
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recognized that these demands create 
workload pressures that it is ill-equipped 
to meet. 


20.104 The Department has not 
identified the incremental costs of the 
New Direction policy or set priorities for 
the spending of existing funds. Funding 
sources include the Pacific Fisheries 
Adjustment and Restructuring Program 
($80 million to be spent from 1999-2000 
to 2001-02 on selective fishing, 
diversification, habitat and adjustment 
assistance projects); the Aboriginal 
Fisheries Strategy (approximately 

$16 million annually); and the 
Canada—U.S. Treaty (a share of the 

$209 million available). The Department 
does not appear to have set priorities for 
the allocation of these funds to the several 
programs it has initiated for habitat 
conservation and protection and for 
adjustment assistance. Funds under the 
Canada—U.S. Treaty are to be 
administered jointly by both countries to 
invest in habitat, stock enhancement, 
science and salmon management. 
However, the identification of stocks at 
risk, which 1s urgently needed to set 
spending priorities, has not yet been 
completed. 


20.105 The regional office has not 
determined how it will cover funding 
shortages in key areas like scientific 
support. We have noted that a risk 
assessment has not been done in order to 
prioritize funds to meet the objectives of 
the New Direction policy, such as 
managing for biodiversity. The 
Department has identified potential 
sources of new funds, such as funding for 
the implementation of the proposed 
endangered species legislation, and is 
counting on obtaining them. However, it 
has not yet assessed how much money it 
must allocate to salmon under the New 
Direction policy, and how much will be 
required for the other species under its 
jurisdiction. 
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20.106 The Department anticipates that 
management costs could be offset in the 
future by partnering with stakeholders 
from the commercial fishery, as indicated 
in its 1996 Fisheries Management 


Partnering Policy Principles. However, The Department has 
the regional office has trouble identifying . on 
costs that are directly attributable to the not identified the new 
fishing industry and therefore potentially costs of doing 
transferable. While it is still unable to : 

business. 


determine costs by species, the regional 
office estimates that approximately 

$85 million of its total budget of 

$230 million in 1998-1999 was spent on 
salmon, including habitat, enhancement, 
science and fisheries management. The 
regional office’s dilemma is that it is 
putting a significant amount of money into 
managing the Pacific salmon commercial 
fishery, for a declining economic return. 


20.107 The Department recognizes that 
the Aboriginal Fisheries Strategy is 
assuming a greater role in fisheries 
management because of the land claims 
treaty process. Treaty settlements will 
provide money to First Nations to 
participate more fully in the management 
of their fisheries. Aboriginal participation 
in fisheries management will have an 
impact on the way the fishery is managed 
in the future. The evolving role of First 
Nations in fisheries management could 
thus be an important agent of change. 
However, we have not yet seen how the 
Department plans to co-ordinate fisheries 
management functions with treaty 
settlements under its goal of establishing 
regional management boards. 


20.108 The Province of British Columbia 
is potentially a large external source of 
funds for achieving habitat conservation 
goals held in common with the 
Department. The Province of British 
Columbia has created its own 
organization, Fisheries Renewal B.C., to 
invest public funds in the revitalization of 
the fishing industry. In 1999, Fisheries 
Renewal B.C. contributed several million 
dollars to projects aimed at restoring 
habitat, diversifying the fisheries and 


External funding 
sources are needed. 
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creating fisheries-related employment. 
The establishment of a joint Fisheries 
Renewal Advisory Board under the 
Canada-British Columbia Agreement on 
the Management of Pacific Salmon 
Fishery Issues would, however, increase 
the effectiveness of the development and 
delivery of both parties’ salmon habitat 
and enhancement programs. 


20.109 Asa basis for setting priorities 
in the allocation of resources to meet the 
demands of the New Direction policy, 
the Department should complete risk 
assessments in areas where management 
information is incomplete or lacking. 


Department’s response: Fisheries and 
Oceans agrees. The salmon assessment 
frameworks being prepared (see response 
to paragraph 20.56) will be critical to 
identify priorities. In addition, the 
Department is initiating a review of the 
salmon management process. Resources, 
both from budget reallocation and from 
other sources, will be directed in 
accordance with priority. 


Conclusion 


20.110 The past management of Pacific 
salmon and its habitat, together with 
factors beyond the Department’s control, 
have resulted in serious declines in many 
stocks. This has posed a threat to the 
future sustainability of the fisheries. The 
Department has responded by releasing a 
new sustainable fisheries policy that 
focusses on conservation and the 
protection of diversity of salmon stocks. It 
is taking action to overhaul its fisheries 
management regime, but needs to 
overcome obstacles such as a limited 
information base and poor relations with 
stakeholders. The Department’s challenge 
is to conserve stocks while providing 
opportunities for harvesting. Appropriate 
action to protect the genetic diversity of 
existing stocks will likely mean that 
fisheries will be curtailed in the short term 
in the interests of rebuilding the resource 
base to sustain fisheries in the future. 
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yy About the Audit 


Objective 


Our audit objective was to determine whether the Pacific salmon fisheries management process and practices 
of the Department of Fisheries and Oceans are ensuring the conservation of the resource base and the 
sustainability of the Aboriginal, commercial and recreational fisheries that depend on it. 


Criteria 
We expected that the Department would have: 


e adopted an appropriate sustainable approach, including acquiring and applying the necessary scientific 
information and contemporary knowledge, to fulfil its mandate to conserve and protect the Pacific 
salmon resource base and, where necessary, restore its abundance and the diversity of stocks; 


e ensured that all sectors were co-ordinating their operations and communications in the development and 
implementation of management plans for the Aboriginal, commercial and recreational fisheries, using 
resource conservation objectives to achieve sustainable development; and 


e negotiated agreements or entered into arrangements with stakeholders, both international and domestic, 
to ensure the conservation and protection of Canada’s Pacific salmon resource and the equitable 
distribution of fish that are surplus to spawning requirements. 


Approach 


Our audit included meetings or conversations with departmental staff in headquarters in Ottawa, in the 
regional office in Vancouver, and in district offices and research facilities. We reviewed Fisheries and Oceans 
files and documentation, and made formal requests for specific information from departmental staff in 
headquarters and the regional office. A large amount of material was obtained through the review of external 
and internal reports. 


Audit Team 


Assistant Auditor General: Don Young 
Principal: Geoffrey L. Robins 


John Sokolowski 
Erika Szenasy 
Shawn Vincent 
Erin Windatt 


For information, please contact Geoffrey Robins. 
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Appendix A 


Coho Case Study — The Development of a Fisheries Management Crisis 


Declines in coho stocks first became apparent in the mid—1970s to mid—1980s 


The Department of Fisheries and Oceans has documented the coho problem since the 1970s. Activity in the coho 
fisheries was intense back then, with exploitation rates often exceeding 70 percent. Declines in wild escapements were 
noted by the Department, particularly in the early-run Skeena River stock. The Department took some measures, such 
as the closure of directed net fisheries in the Skeena and Fraser Rivers, but otherwise coho fisheries were not closely 
regulated. Coho research focussed on freshwater habitat and environmental concerns, as it was widely believed that 
coho was a suitable indicator species of habitat impacts. 


Further declines in coho stocks in the mid—1980s through the mid—1990s increased concern about their 
conservation 


The Department noted that the abundance of wild coho was clearly declining throughout southern British Columbia 
during this period. Declines in the upper Skeena River were described as a conservation concern. However, assessment 
advice based on information at the time indicated that the aggregate stock in southern British Columbia was only 
moderately over-exploited. 


Catch and escapement records revealed the dramatic decline in wild coho. For example, the Department noted that 
recreational and commercial catches of wild coho dwindled by an average of about 66,000 fish per year between 1976 
and 1989. Overfishing and habitat loss were believed to be responsible for this decline. The proportion of Strait of 
Georgia coho being caught (the exploitation rate) averaged 10 percent higher than the 65 to 70 percent considered 
optimal for sustaining stocks. 


During the late 1980s there were increasing concerns about coho abundance in populations around the Strait of 
Georgia and in the upper Skeena River. There was a growing recognition by the Department that exploitation rates 
nearing 80 percent of southern British Columbia stocks were not sustainable. These concerns led to the initiation of a 
coho conservation program in southern British Columbia and some further management actions in northern sockeye 
fisheries. 


In 1989, the Department formed the Coho Steering Committee to address this major problem of declining coho stocks. 
The wild coho rebuilding plan was designed to balance environmental effects through a program that would integrate 
habitat management, harvest management and enhancement. Together with fishery advisors, the Coho Steering 
Committee began to develop a plan for rebuilding wild coho stocks. However, in 1992 the Committee stated that wild 
coho stocks in the Strait of Georgia and the Fraser River were still declining. 


In 1995 the Department’s scientists warned managers of serious problems with coho 


It was not until 1995 that departmental managers began to pay serious attention to coho stocks off the south coast of 
British Columbia. Even then, not enough was done to address the rapid declines in survival rates. For example, 
exploitation rates for coho stocks in the south of the province were 70 to 80 percent before 1994. In 1995, the rate was 
reduced to 60 percent. 


The Department’s Pacific Stock Assessment Review Committee (PSARC) recommended that the exploitation rates for 
Strait of Georgia coho be cut from between 65 and 70 percent to below 65 percent for the 1996 fishing season. Fishery 
managers were warned of continuing low survival rates for Strait of Georgia coho stocks. 


The PSARC continued to warn management about the low marine survival rate expected to continue in 1997. Some 
coho stocks were not expected to achieve conservation goals, even with no fishing. The PSARC urged caution in 
exploiting southern British Columbia coho stocks. As a result of the advice, there was no directed commercial coho 
fishery in southern British Columbia for the 1997 fishing season. In the north, the harvest rate was reduced to 

53 percent. The recreational fishery remained open with reduced daily catches of coho. These actions brought the 
exploitation rate for coho down to between 20 and 40 percent overall. 
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Calls in 1998 for drastic action to conserve coho 


The PSARC advised managers to use extreme caution in applying forecasts of coho abundance in 1998. The Skeena 
and Thompson River coho stock aggregates were extremely depressed, and would continue to decline under current 
marine survival conditions even in the absence of any fishing mortality. Some individual spawning populations were at 
high risk of biological extinction. The status of the Strait of Georgia and lower Fraser stock aggregate was also 
deteriorating because of continuing low marine survival rates. 


Action finally taken to conserve existing stocks and develop a recovery plan for those stocks at risk 


On 19 June 1998 the Minister of Fisheries and Oceans recognized that a coho crisis existed. He announced the Salmon 
Management Plan to respond to the decline in abundance. The Plan was based on protecting upper Skeena and 
Thompson River coho. Fishing was stringently limited in areas where these coho were prevalent, in effect closing the 
recreational and commercial coho fishery. The new policy, A New Direction for Canada’s Pacific Salmon Fisheries, 
makes selective fishing mandatory. If this is ineffective, fisheries will be closed. 


Source: Based on documentation of the Department of Fisheries and Oceans and the Pacific Fisheries Resource Conservation Council. 
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Appendix B 


Conditions Necessary to Achieve Sustainable Fisheries 


e Anecosystem-based management system has been adopted. As a result, the watershed is the essential stewardship 
unit and local watershed interests are directly involved in fisheries and environmental management decisions. 


e Salmon escapement goals (how many fish must return to a stream to sustain the run) are established by using a 
habitat-based conservation approach and consider non-human uses of salmon and the ecosystem (biodiversity, 
wildlife, etc.). In addition, managers recognize the inherent variability in the productive capacity of freshwater, 
estuarine, and marine habitats, and the impact of this variability on salmon populations. 


e Resource managers have adopted a risk averse approach to harvest management, which minimizes the potential 
for not achieving escapement goals. 


e Salmon fisheries have been restructured to reduce mixed-stock fisheries, increase terminal fisheries, emphasize 
selective harvesting methods, and expand stock-specific fisheries. 


e Decisions on the allocation of surplus salmon and steelhead production are independent from biological decisions 
that establish the total allowable harvest of these resources. 


e The emphasis of artificial propagation has shifted from salmonid production (to support harvests) toward salmonid 
restoration (helping to rebuild weaker stocks). 


e The social and political environment has changed to focus decision-making authority in watersheds, to provide 
incentives for stewardship initiatives, and to establish regional and international oversight for salmon production. 


Source: Sustainable Fisheries Foundation. Towards Sustainable Fisheries: Building a Cooperative Strategy for Balancing the Conservation and Use of 
West Coast Salmon and Steelhead Populations. 1996. 
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